Uploading Multipart files

[guneysus]

I can not upload files with restless.
It raises exception before my Resource's def create(*args ,**kwargs) method and seems to serialize file content as json.
I am using httpie for testing uploads from CLI.

https://github.com/jkbrzt/httpie#file-upload-forms

➜  blog git:(master) ✗ http -f POST :8888/api/snippets file@test.txt -p Hh 
POST /api/snippets HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate, compress
Content-Length: 188
Content-Type: multipart/form-data; boundary=35eaef06eea1424184e438ce4d907ed3
Host: localhost:8888
User-Agent: HTTPie/0.8.0

HTTP/1.1 500 Internal Server Error
Content-Encoding: gzip
Content-Length: 1130
Content-Type: application/json; charset=UTF-8
Date: Fri, 11 Mar 2016 13:24:44 GMT
Server: TornadoServer/4.3
Vary: Accept-Encoding

Traceback

Traceback (most recent call last):
  File "/home/ahmed/opt/pycharm-5.0.4/helpers/pydev/pydevd.py", line 2411, in <module>
    globals = debugger.run(setup['file'], None, None, is_module)
  File "/home/ahmed/opt/pycharm-5.0.4/helpers/pydev/pydevd.py", line 1802, in run
    launch(file, globals, locals)  # execute the script
  File "/home/ahmed/PycharmProjects/blog/application.py", line 55, in <module>
    main()
  File "/home/ahmed/PycharmProjects/blog/application.py", line 50, in main
    tornado.ioloop.IOLoop.current().start()
  File "/home/ahmed/PycharmProjects/blog/venv/local/lib/python2.7/site-packages/tornado/ioloop.py", line 883, in start
    handler_func(fd_obj, events)
  File "/home/ahmed/PycharmProjects/blog/venv/local/lib/python2.7/site-packages/tornado/stack_context.py", line 275, in null_wrapper
    return fn(*args, **kwargs)
  File "/home/ahmed/PycharmProjects/blog/venv/local/lib/python2.7/site-packages/tornado/netutil.py", line 274, in accept_handler
    callback(connection, address)
  File "/home/ahmed/PycharmProjects/blog/venv/local/lib/python2.7/site-packages/tornado/tcpserver.py", line 269, in _handle_connection
    future = self.handle_stream(stream, address)
  File "/home/ahmed/PycharmProjects/blog/venv/local/lib/python2.7/site-packages/tornado/httpserver.py", line 180, in handle_stream
    conn.start_serving(self)
  File "/home/ahmed/PycharmProjects/blog/venv/local/lib/python2.7/site-packages/tornado/http1connection.py", line 693, in start_serving
    self._serving_future = self._server_request_loop(delegate)
  File "/home/ahmed/PycharmProjects/blog/venv/local/lib/python2.7/site-packages/tornado/gen.py", line 282, in wrapper
    yielded = next(result)
  File "/home/ahmed/PycharmProjects/blog/venv/local/lib/python2.7/site-packages/tornado/http1connection.py", line 706, in _server_request_loop
    ret = yield conn.read_response(request_delegate)
  File "/home/ahmed/PycharmProjects/blog/venv/local/lib/python2.7/site-packages/tornado/http1connection.py", line 151, in read_response
    return self._read_message(delegate)
  File "/home/ahmed/PycharmProjects/blog/venv/local/lib/python2.7/site-packages/tornado/gen.py", line 294, in wrapper
    Runner(result, future, yielded)
  File "/home/ahmed/PycharmProjects/blog/venv/local/lib/python2.7/site-packages/tornado/gen.py", line 956, in __init__
    self.run()
  File "/home/ahmed/PycharmProjects/blog/venv/local/lib/python2.7/site-packages/tornado/gen.py", line 1017, in run
    yielded = self.gen.send(value)
  File "/home/ahmed/PycharmProjects/blog/venv/local/lib/python2.7/site-packages/tornado/http1connection.py", line 238, in _read_message
    delegate.finish()
  File "/home/ahmed/PycharmProjects/blog/venv/local/lib/python2.7/site-packages/tornado/httpserver.py", line 291, in finish
    self.delegate.finish()
  File "/home/ahmed/PycharmProjects/blog/venv/local/lib/python2.7/site-packages/tornado/web.py", line 2022, in finish
    self.execute()
  File "/home/ahmed/PycharmProjects/blog/venv/local/lib/python2.7/site-packages/tornado/web.py", line 2055, in execute
    **self.path_kwargs)
  File "/home/ahmed/PycharmProjects/blog/venv/local/lib/python2.7/site-packages/tornado/gen.py", line 282, in wrapper
    yielded = next(result)
  File "/home/ahmed/PycharmProjects/blog/venv/local/lib/python2.7/site-packages/tornado/web.py", line 1443, in _execute
    result = method(*self.path_args, **self.path_kwargs)
  File "/home/ahmed/PycharmProjects/blog/venv/local/lib/python2.7/site-packages/tornado/gen.py", line 282, in wrapper
    yielded = next(result)
  File "/home/ahmed/PycharmProjects/blog/venv/local/lib/python2.7/site-packages/restless/tnd.py", line 40, in _method
    yield self.resource_handler.handle(self.__resource_view_type__, *args, **kwargs)
  File "/home/ahmed/PycharmProjects/blog/venv/local/lib/python2.7/site-packages/tornado/gen.py", line 282, in wrapper
    yielded = next(result)
  File "/home/ahmed/PycharmProjects/blog/venv/local/lib/python2.7/site-packages/restless/tnd.py", line 168, in handle
    raise gen.Return(self.handle_error(err))
  File "/home/ahmed/PycharmProjects/blog/venv/local/lib/python2.7/site-packages/restless/resources.py", line 315, in handle_error
    return self.build_error(err)
  File "/home/ahmed/PycharmProjects/blog/venv/local/lib/python2.7/site-packages/restless/tnd.py", line 160, in handle
    self.data = self.deserialize(method, endpoint, self.request_body())
  File "/home/ahmed/PycharmProjects/blog/venv/local/lib/python2.7/site-packages/restless/resources.py", line 337, in deserialize
    return self.deserialize_list(body)
  File "/home/ahmed/PycharmProjects/blog/venv/local/lib/python2.7/site-packages/restless/resources.py", line 351, in deserialize_list
    return self.serializer.deserialize(body)
  File "/home/ahmed/PycharmProjects/blog/venv/local/lib/python2.7/site-packages/restless/serializers.py", line 62, in deserialize
    return json.loads(body.decode('utf-8'))
  File "/usr/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 384, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded", "error": "No JSON object could be decoded"}

[guneysus]

I marked lib/python2.7/site-packages/restless/serializers.py:62 as breakpoint and
the body variable is below:

'--8d1e7a823982469d82c0358769ccba4c
Content-Disposition: form-data; name="file"; filename="test.txt"

Test.txt content

--8d1e7a823982469d82c0358769ccba4c--
'

class JSONSerializer(Serializer):
    def deserialize(self, body):
        """
        The low-level deserialization.

        Underpins ``deserialize``, ``deserialize_list`` &
        ``deserialize_detail``.

        Has no built-in smarts, simply loads the JSON.

        :param body: The body of the current request
        :type body: string

        :returns: The deserialized data
        :rtype: ``list`` or ``dict``
        """
        if isinstance(body, bytes):
            return json.loads(body.decode('utf-8')) # LINE 62
        return json.loads(body)

[guneysus]

I implemented a custom multipart file aware serializer for Restless Tornado.

https://gist.github.com/guneysus/3604327fe9946618ebdc

[psychofisch]

4 years and this is still open. So it is still impossible to upload files using this API, right?