Skip to content

Tries to perform "utf8" decoding when encoding set to "base64" on file commit #427

Closed

Description

@JonBoyleCoding

When I'm trying to commit a binary file into a repository and setting the encoding to "base64" (as specified in https://docs.gitlab.com/ce/api/repository_files.html#create-new-file-in-repository), I'm getting the following error:

Traceback (most recent call last):
File "create_submission_point_for_group.py", line 261, in
project.files.create(blob)
File "/usr/local/lib/python2.7/dist-packages/gitlab/cli.py", line 42, in wrapped_f
return f(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/gitlab/exceptions.py", line 247, in wrapped_f
return f(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/gitlab/v4/objects.py", line 1806, in create
server_data = self.gitlab.http_post(path, post_data=new_data, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/gitlab/init.py", line 792, in http_post
post_data=post_data, files=files, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/gitlab/init.py", line 678, in http_request
prepped = self.session.prepare_request(req)
File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 394, in prepare_request
hooks=merge_hooks(request.hooks, self.hooks),
File "/usr/lib/python2.7/dist-packages/requests/models.py", line 298, in prepare
self.prepare_body(data, files, json)
File "/usr/lib/python2.7/dist-packages/requests/models.py", line 424, in prepare_body
body = complexjson.dumps(json)
File "/usr/lib/python2.7/dist-packages/simplejson/init.py", line 380, in dumps
return _default_encoder.encode(obj)
File "/usr/lib/python2.7/dist-packages/simplejson/encoder.py", line 291, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/usr/lib/python2.7/dist-packages/simplejson/encoder.py", line 373, in iterencode
return _iterencode(o, 0)

This makes it seem like the python api is ignoring the encoding option. I've ensured the encoding is set to base64 in the 'blob' that I've passed to project.files.create(). Other ordinary files without the encoding commit perfectly fine.

Activity

  1. JonBoyleCoding commented on Feb 21, 2018

    @JonBoyleCoding
    Author

    After a long look around I found that you actually have to use the package base64 to encode the file, something like below

    encoding = "text"
    content = open(f,"r").read()
    
    if IsBinaryFile(f): # Uses pip package binaryornot to determine if it's a binary file
    	encoding = "base64"
    	content = base64.b64encode(content)
    
    # Add to the data blob of all files being added
    data["actions"].append({
    	"action": "create",
    	"file_path": filename,
    	"encoding": encoding,
    	"content": content
    })
    

    I might suggest that something should be added to commits documentation describing how to handle the encoding, as it's not particularly clear for someone that's new to how the API works. Here's the page I mean:

    http://python-gitlab.readthedocs.io/en/stable/gl_objects/commits.html

  2. gpocentek commented on Feb 22, 2018

    @gpocentek
    Contributor

    Hi @JonBoyleCoding

    Sorry for my late answer, and thanks a lot for your feedback on this issue. I'll update the documentation to add the information you provided.

  3. added this to the v1.4 milestone on Feb 22, 2018
  4. locked as resolved and limited conversation to collaborators on Nov 9, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions