Tuesday, 17 June 2014

Common porting errors wiki

I think it could be helpful to someone (at the very least it would be helpful to future me) if i mention here the problems and errors i faced in porting sugar to python3 and their solution according to me.
So here are the errors: 

SyntaxError: invalid syntax
Line of error: except Exception, e:
Explanation: The correct syntax for exception in Python3 use as instead of a comma
Correct code: except Exception as e:

ImportError: No module named 'urllib2'
Line of error: import urllib2
Explanation: In Python3 urllib2 has been merged with urllib , so there is no module named urllib2. You can directly import the required module like urllib.request or urllib.error .
Correct Code: import urllib.request, urllib.error, urllib.parse

ImportError: No module named 'StringIO'
Line of error: import StringIO
Explanation : The StringIO and cStringIO modules are gone. Instead, import the io module and use io.StringIO or io.BytesIO for text and data respectively.
Correct code: import io

ValueError: can't have unbuffered text I/O
Line of Error: sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0)
Explanation: As the error says, we cannot have unbuffered text I/O in Python3. So i have removed this line for now, although there are some other ways to get unbuffered text I/O like using the -u argument.
Correct Code:

/usr/bin/env: python3: No such file or directory
Explanation : On some debugging, i found out that in my case the error was due to the virtualenv . As the virtualenv was running using python2 , it couldn't find python3 inside the virtual environment. So this error may vary from case to case and we would need to find out why the interpreter cannot find the python3 directory
Line of error: subprocess.check_call(["python2.7",
                           os.path.join(source_dir, "virtualenv.py"),
                           "-q", "--system-site-packages",
Correct Code: subprocess.check_call(["python3.3",
                           os.path.join(source_dir, "virtualenv.py"),
                           "-q", "--system-site-packages",

TypeError: Unicode-objects must be encoded before hashing
 Line of Error: path_hash.update(self._config_path)
Explanation: Unlike Python2, in Python3 all the hashlib fucntions take in bytes rather than strings so we need to encode the string before passing.
Correct Code: path_hash.update((self._config_path).encode('utf-8'))

TypeError: expected bytes, bytearray or buffer compatible object
Line of error: base64_hash = base64_hash.replace("+", "0")
Explanation: The .replace method requires a string input , hence we need to decode what we encodeed in the previous error.
Correct Code: base64_hash = base64_hash.decode('utf-8')
                        base64_hash = base64_hash.replace("+", "0")

TypeError: Type str doesn't support the buffer API
Line of Error: for mounted in mount_output.split("\n"):
Explanation: This is also a type error like the previous one which means we are again confusing bytes and strings. On debugging, i found out that mount_output is actually a byte object and thus string method split is not compatible with it, so we first need to decode it.
Correct Code: mount_output = mount_output.decode('utf-8')
                        for mounted in mount_output.split("\n"):

SyntaxError: assignment to keyword
Line of Error: False, True = 0, 1
Explanation: In Python3 True and False are keywords instead of literals , so they cannot be reassigned.
Refer to this article for the history of True/False keywords: http://python-history.blogspot.in/2013/11/story-of-none-true-false.html
Correct code: Just remove this line because True and False are already defined as keywords

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 79: ordinal not in range(128)
Explanation: seems to be another one of those strings vs unicode vs bytes problem.
Correct code : b'' instead of ''

AttributeError: 'module' object has no attribute 'find'
Line Of Error: if string.find(thing, '=') >= 0:
Explanation: The string.find method has been deprecated in python3 , use the .find() method of strings instead
Correct code: if thing.find('=') >= 0:

FileNotFoundError: [Errno 2] No such file or directory: '/home/curiousguy13/py3sugar/sugar-build/build/commands/common.pyc'
Line Of Error: os.unlink(os.path.join(common.commands_dir, "common.pyc"))
Explanation: In python3 the the compiled header files are stored in __pycache__ instead of the same directory. see more here : http://stackoverflow.com/questions/154443/how-to-avoid-pyc-files
So, just changing the directory address to __pycache__ in common.py solved the issue. 

PS: The list is not exhaustive and i will keep on adding new errors and problems that i face with their solutions

WARNING: Use the above post at your own risk. The explanation and the solution of errors are according to my own knowledge of Python and some online research and they might be wrong in which case you can point out the mistake in the comments and i can correct it.


  1. Great! This is very useful.
    Are you implementing changes compatibles with python 2.7 and 3?

    1. As far as i know, sugar-toolkit is responsible for running the activities, so i am making changes compatible with Python3 only in sugar-build and sugar whereas i intend to make sugar-toolkit compatible with python2.7 and 3.


Math of Intelligence : Logistic Regression

Logistic Regression Logistic Regression ¶ Some javascript to enable auto numberi...