Transform: Convert data

Exploits need to manipulate data easily: encode it into base64, compute its hash, HTML-decode it, etc. That's what transform is for.

Each category of transformation is available as a submodule of transform. When you do from ten import *, transform is imported as tf, and the most used modules, such as json or base64, are included as root modules.

Each category of transformation is available as a submodule of transform.

>>> from tenlib.transform import base64, html
>>> base64.decode('dGVzdA==')
b'test'
>>> html.decode('<div>test</div>')
'<div>test</div>'

Methods support both bytes and str as input, as well as lists and dicts. Refer to Multiform for more details.

Here are a few examples for the most common transformations.

JSON

json.encode({"a": "b", "c": "d"})

'{"a":"b","c":"d"}'

json.decode('{"a": "b", "c": "d"}')

{"a": "b", "c": "d"}

Base 64 encoding (base64)

base64.encode('test')

'dGVzdA=='

base64.decode('dGVzdA==')

b'test'

Query string (qs)

Parse query string and URL-encode/decode.

qs.parse('k1=v1&k2=v2')

{'k1': 'v1', 'k2': 'v2'}

qs.unparse({'k1': 'v1', 'k2': 'v2'})

'k1=v1&k2=v2'

qs.decode('%41%42%43%3a%44%45%46')

'ABC:DEF'

qs.encode('ABC:DEF')

'ABC%3aDEF'

qs.encode_all('ABC:DEF')

'%41%42%43%3a%44%45%46'

Table

Convert data formatted into a table (such as CSV) to a list (of list), and back. This example converts an array of colon separated lines into a table.

table.split('''\
username:password:email
admin:dUD6s55:admin@site.com
moderator:123456!:moderator@site.com
user:Password1:user@gmail.com\
''', '\n', ':')

[
    [b'username', b'password', b'email'],
    [b'admin', b'dUD6s55', b'admin@site.com'],
    [b'moderator', b'123456!', b'moderator@site.com'],
    [b'user', b'Password1', b'user@gmail.com'],
]

table.join([
    [b'username', b'password', b'email'],
    [b'admin', b'dUD6s55', b'admin@site.com'],
    [b'moderator', b'123456!', b'moderator@site.com'],
    [b'user', b'Password1', b'user@gmail.com'],
], '\n', ':')

'''\
username:password:email
admin:dUD6s55:admin@site.com
moderator:123456!:moderator@site.com
user:Password1:user@gmail.com\
'''

Hashing

hashing.md5('test')

'098f6bcd4621d373cade4e832627b4f6'

hashing.sha1('test')

'a94a8fe5ccb19ba61c4c0873d391e987982fbbd3'

Others

Lots of other transforms are available. Refer to the documentation for more details.

Multiform

As you may have seen, you can feed either bytes or strings to the conversion functions, and they will handle the convertion into the proper type. This behaviour also works with dicts and lists:

base64.encode('test')

'dGVzdA=='

base64.encode(b'test')

'dGVzdA=='

base64.encode([
    'test',
    b'test2',
    'test3'
])

[
    'dGVzdA==',
    'dGVzdDI=',
    'dGVzdDM='
]

qs.decode({
    'k1': '%41%42%43',
    'k2': '%44%45%46'
})

{
    'k1': 'ABC',
    'k2': 'DEF'
}

CLI: the tf program

The tf program makes transforms available from the CLI as well:

$ echo 'dGVzdA==' | tf base64.decode
test

The filters can be chained. Here, we convert JSON into an URL-encoded string:

$ echo '{"a":"b","c":"d"}' | tf json.decode qs.unparse
a=b&c=d

Check the --help for further details.