Making CharLCD compatible with Python3

We have working LCD and LCD Manager library. They works on Python 2.7 but not v3. I thought about making them v3 compatible but before that I had to upgrade my system. And this was bad idea πŸ™‚

Problem with SD card and free space forced me to change system drive to USB. And then everything went from bad to worse πŸ™‚ But after 5 hours system was back to being ok.

As first we need to port CharLCD. After reading https://docs.python.org/3/howto/pyporting.html I started porting.

I went with futurize library (http://python-future.org/automatic_conversion.html), installation by PIP

pip install future

After that I run:

futurize --stage1 -w **/*.py

It done small changes to my code and tests were still passing, that is is good. Piader was working.
Next step:

futurize --stage2 -w **/*.py

Another pack of changes, p2 test passing and forcing test to run with python3 with:

python3 -m nose --with-specplugin

shows green:) Nice.
Started Piader and it was looking good but when I pressed a key game crashed.
Problem was with sending data over socket. Python3 require byte while python2 can go with string.
First attempt to fix was by adding byte conversion and it worked in p3 but failed in p2.
To fix this I used call do decode/encode. In local_key.py changed send function to

self.conn_socket.send(event.encode('UTF-8'))

And reciving in event_server.py changed to

data = self.client.recv(BUFFER_SIZE).decode("UTF-8")

This is working in both p2 and p3.

Next step is to look at all changes and improve things manually. I changed one line, it was dividing, I replaced old_div with // in game.py

Stop! Jenkins time!

And this is hard part. We want Jenkins for both environments. I found tox library and ShiningPanda plugin. It looks promising. Main question: what is important ? For me tests and violation report. It would be nice to have cpd and coverage.

sudo pip install future

Next I followed this: http://shiningpanda.com/tox-on-jenkins-with-shiningpanda-plugin.html

I created new project but made a mistake, created project with space in name… never do it.
I quickly configured it as tutorial says.
First run and first problem, tests were not found. Path entered in tox.ini was wrong, changed it to charlcd/tests.
Another run and it crashed again, missing mock package. Added it to deps in tox.ini.
And another run another problem. This time builtins missing. Added another deps: future.
Run and this time p27 passed but p34 failed. Error was ERROR: InterpreterNotFound: python3.3. It is not good 😦 Tox is looking for python 3.3 i have 3.4 installed. Shining panda in options has only 3.3. What now ? I entered project configuration at Jenkins and added py34 in Tox / Custom environments. Run again and.. it worked!
We have now a dual environment with nose tests. Lets try and add violation reports.
I copied entry from old project:

rm -f pylint.log
for f in `find . -name '*.py'|egrep -v '^./tests/'`; do
pylint --msg-template="{path}:{line}: [{msg_id}({symbol}), {obj}] {msg}" --reports=y $f >> pylint.log
done || :

First run and so many violations.. this script looks in all files in current directory but in current directory we have a virtual environment πŸ™‚ So all files were checked πŸ™‚ Quick command modification:

rm -f pylint.log
for f in `find ./charlcd/ -name '*.py'|egrep -v '/tests/'`; do
pylint --msg-template="{path}:{line}: [{msg_id}({symbol}), {obj}] {msg}" --reports=y $f >> pylint.log
done || :

And we are checking only our package and without tests files. Now it looks nice. But this approach is wrong! We are launching pylint not from virtual envs. Bad idea.
I tried to move pylint to tox.ini but after 3 hours I gave up. Next I tried with flake8 it was much worse.. I read all tutorials and I had exactly same code but my was failing.
I almost gave up but then I done something stupid πŸ™‚ and it worked πŸ™‚ I have flakes8 checking my code.
And I must say it is much more demanding then pylint.
After long day I ended with such tox.ini:

[tox]
envlist = py27,py34
skipsdist = True
[testenv]
setenv = VIRTUAL_ENV={envdir}
deps=
    nose
    mock
    future
    flake8
whitelist_externals = /bin/bash
commands= nosetests --with-xunit --xunit-file=junit-{envname}.xml charlcd/tests
    /bin/bash -c "flake8 --output-file=flake8-{envname}.log charlcd || :"
[flake8]
show-source = True
exclude = .git,.venv,.tox,dist,doc,build,*egg,*/tests/*

I also learned that I have much to learn πŸ™‚
I wounder if I can enable pylint.. lets see..

 /bin/bash -c "pylint charlcd >> pylint-{envname}.log || :"

And pylint is back..All in all the code is in quite mess. Time to clean this.
Strange thing happens, with each commit pylint reported more and more problems. Reason was quite funny πŸ™‚

pylint charlcd >> pylint-{envname}.log

See >> ? It’s appending πŸ™‚ when changed to > violations were back to normal.

It took some time but now flake8 reports 0 violations but pylint reports 14. For now I see no way of going lower.

Summary

We have working CharLCD package in both Python 2.7 and Python 3.4. And we have Jenkins doing tests and validation in dual-environment.
Final word: porting to Python3 is easy there are lots of tools and articles.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s