Doton the Control Node – part 2 – weather widget

Long, long time ago in a post far, far away we made a weather worker (part 1, part 2). It is time to find it and add to our project along with new widget.

And as a side effect, we will sugarcoat GUI a little bit.

Doton part 1
Doton part 3

Source code @ GitHub

Reviving a legacy code

I copied code from an endless hole of old code and looked how it works.
Seems we have a threaded service to read data from Openweathermap. It takes places dictionary in init to know our places of interest.
This service is started and handled by a worker.
Ok, let’s refactor it and use in Doton project.
Almost all went well. I had to change urllib2 to urllib, add an API key and fix indexes.
All in all, we have two new files in the project, an Openweather service:

"""Openweathermap service"""
import time
import threading
import urllib.error
import urllib.request
import json
import datetime
import socket


class Openweather(threading.Thread):
    """Openweather class to get data from openweathermap"""
    def __init__(self, cities, apikey):
        threading.Thread.__init__(self)
        self.cities = cities
        self.work = True
        self.tick = {'sleep': 10, 'weather_counter': 6*10, 'forecast_counter': 6*60}
        self.tick['_wcounter'] = self.tick['weather_counter'] + 1
        self.tick['_fcounter'] = self.tick['forecast_counter'] + 1

        self.weather_row_stub = {
            'update': 0,
            'temperature_min': 0,
            'temperature_max': 0,
            'temperature_current': 0,
            'humidity': 0,
            'pressure': 0,
            'wind_speed': 0,
            'wind_deg': 0,
            'clouds': 0,
            'weather_id': 0,
            'weather': ''
        }
        self.current_weather_raw = {i: "" for i in cities}
        self.current_weather = {i: self.weather_row_stub.copy() for i in cities}

        self.forecast_weather_raw = {i: {} for i in cities}
        self.forecast_weather = {i: {} for i in cities}

        self.url_current = "http://api.openweathermap.org/data/2.5/weather?id=%CITY_ID%&units=metric&mode=json&APPID="+apikey
        self.url_forecast = "http://api.openweathermap.org/data/2.5/forecast/daily?id=%CITY_ID%&mode=json&units=metric&cnt=4&APPID="+apikey

    def run(self):
        """main loop, reads data from openweather server"""
        while self.work:
            if self.tick['_wcounter'] > self.tick['weather_counter']:
                for city_id in self.cities:
                    """current weather"""
                    url = self.url_current.replace("%CITY_ID%", str(city_id))
                    json_data = self._fetch_data(url)
                    if json_data:
                        self.current_weather_raw[city_id] = json_data
                        self._decode(city_id)
                        self.tick['_wcounter'] = 0

            if self.tick['_fcounter'] > self.tick['forecast_counter']:
                for city_id in self.cities:
                    """ forecast """
                    url = self.url_forecast.replace("%CITY_ID%", str(city_id))
                    json_data = self._fetch_data(url)
                    if json_data:
                        for row in json_data['list']:
                            date = (datetime.datetime.fromtimestamp(int(row['dt']))).strftime("%Y-%m-%d")
                            self.forecast_weather_raw[city_id][date] = row
                            self.forecast_weather[city_id][date] = self._decode_forecast(row)
                        self.tick['_fcounter'] = 0

            time.sleep(self.tick['sleep'])
            self.tick['_wcounter'] += 1
            self.tick['_fcounter'] += 1

    def stop(self):
        """stops a thread"""
        self.work = False

    def get_raw_data(self, city_id=None):
        """return raw weather data"""
        if city_id is None:
            return self.current_weather_raw.itervalues().next()
        else:
            return self.current_weather_raw[city_id]

    def weather(self, city_id=None):
        """return decoded weather"""
        if not city_id:
            city_id = list(self.cities.keys())[0]

        return self.current_weather[city_id]

    def forecast(self, city_id=None, date=None):
        """return forecast"""
        if not city_id:
            city_id = list(self.cities.keys())[0]
        if date is None:
            date = time.strftime("%Y-%m-%d", time.localtime(time.time() + 24*3600))

        if not date in self.forecast_weather[city_id]:
            return self.weather_row_stub

        return self.forecast_weather[city_id][date]

    def _fetch_data(self, url):
        """fetch json data from server"""
        try:
            request = urllib.request.Request(url, None, {'User-Agent': 'RaspberryPI / Doton'})
            response = urllib.request.urlopen(request)
            data = response.read()
            json_data = json.loads(data.decode())
        except urllib.error.URLError as e:
            json_data = None
            print(time.strftime("%Y-%m-%d %H:%M:%S"), "error fetching from url", url, "\nreason", e.reason)
        except socket.timeout as e:
            json_data = None
            print(time.strftime("%Y-%m-%d %H:%M:%S"), "time out error from url", url, "\nreason", e.reason)
        except ValueError as e:
            json_data = None
            print("Decode failed")

        return json_data

    def _decode(self, city_id):
        """decode raw readings"""
        self.current_weather[city_id] = {
            'temperature_current': self.current_weather_raw[city_id]['main']['temp'],
            'humidity': self.current_weather_raw[city_id]['main']['humidity'],
            'pressure': self.current_weather_raw[city_id]['main']['pressure'],
            'wind_speed': self.current_weather_raw[city_id]['wind']['speed'],
            'wind_deg': self.current_weather_raw[city_id]['wind']['deg'],
            'weather_id': self.current_weather_raw[city_id]['weather'][0]['id'],
            'weather': 'no', #self._weather_codes(self.current_weather_raw[city_id]['weather'][0]['id']),
            'clouds': self.current_weather_raw[city_id]['clouds']['all'],
            'update': self.current_weather_raw[city_id]['dt']
        }

    def _decode_forecast(self, row):
        """decode raw readings"""
        return {
            'temperature_min': row['temp']['min'],
            'temperature_max': row['temp']['max'],
            'humidity': row['humidity'],
            'pressure': row['pressure'],
            'wind_speed': row['speed'],
            'wind_deg': row['deg'],
            'weather_id': row['weather'][0]['id'],
            'weather': 'no', #self._weather_codes(row['weather'][0]['id']),
            'clouds': row['clouds']
        }

It is a legacy code refactored to Python3. And we have a worker:

from service.openweather import Openweather
 
 
class OpenweatherWorker(object):
    def __init__(self, apikey):
        self.cities = {
            3103402: 'Bielsko-BiaΕ‚a',
            2946447: 'Bonn'
        }
        self.handler = Openweather(self.cities, apikey)
 
    def start(self):
        """start worker"""
        self.handler.start()
 
    def shutdown(self):
        """shudown worker"""
        self.handler.stop()
 
    def weather(self, city_id=None, key=None):
        """return curent weather"""
        if key is None:
            return self.handler.weather(city_id)
        return self.handler.weather(city_id)[key]
 
    def forecast(self, city_id=None, forecast_date=None):
        """return forecast"""
        return self.handler.forecast(city_id, forecast_date)

See that worker has hard-coded cities, bad idea. We will change it.
To check how it works I used following script:

from worker.openweather import OpenweatherWorker
import time
 
if __name__ == '__main__':
    w = OpenweatherWorker('my-secret-api-key')
    print("start")
    w.start()
    try:
        while 1:
            d = w.weather()
            print("current weather :", d)
            d = w.forecast()
            print("forecast :", d)
            print("-----------------")
            time.sleep(5)
    except KeyboardInterrupt:
        print("closing...")
    except:
        raise
    finally:
        w.shutdown()

With the following result:

current weather : {'wind_deg': 80, 'weather': 'broken clouds', 'pressure': 1021, 'update': 1493550000, 'weather_id': 803, 'temperature_current': 9.54, 'humidity': 75, 'clouds': 75, 'wind_speed': 2.1}
forecast : {'temperature_max': 12.95, 'weather': 'few clouds', 'temperature_min': 1.27, 'pressure': 975.13, 'weather_id': 801, 'wind_deg': 89, 'humidity': 73, 'clouds': 20, 'wind_speed': 4.01}
-----------------

Good! Something is working.

Widget and tile

Time to think how our tile will look alike. We need information about current/forecasted temperature,Β Β  pressure, humidity, clouds, wind speed, wind direction… What about the weather description? That are freaking 14 data sources…on one tile… no way 😦
What can we do? Maybe two tiles? Or one wide πŸ™‚ That is still 7 info on one tile.
The pressure, temperature and the wind are numeric but humidity and clouds are percentages, they may be represented as circles with proper filling. And weather can be a background image. It would look nice but require huge amount of work, we need 54 images πŸ˜€

Ok, much to do. Let’s start from the small things, the tile with current pressure, temperature and the wind. Those are two short numbers and one big number. To fit this we need a smaller font, something like this:

Let’s prepare a weather widget. It is gonna be a two-tiled one and we will pass coordinates for each tile in a list. We need to store current readings and previous readings so we know what has changed.

class OpenweatherWidget(Widget):
    """Openweathermap widget"""
    def __init__(self, coords, lcd, fonts):
        self.current_widget_pos = coords[0]
        self.forecast_widget_pos = coords[1]
        self.lcd = lcd
        self.fonts = fonts
        self.colours = {
            'background_current': (255, 128, 255),
            'background_forecast': (255, 250, 128),
            'digit_background': (0, 0, 0),
            'border': (244, 244, 244)
        }
        self.current_weather = {
            'current': {'pressure': 1018, 'temperature_current': 11.56, 'humidity': 70, 'wind_speed': 9.3, 'clouds': 75, 'weather_id': 803, 'update': 1493631000, 'wind_deg': 70, 'weather': 'broken clouds'},
            'previous': None
        }
        self_forecast_weather = {
            'current': {'pressure': 975.42, 'clouds': 24, 'temperature_max': 17.09, 'wind_speed': 1.86, 'wind_deg': 82, 'temperature_min': 3.52, 'weather_id': 501, 'humidity': 72, 'weather': 'moderate rain'},
            'previous': None
        }

We mocked reading. Next, goes tiles rendering:

    def draw_widget(self):
        """draw a tiles"""
        self._draw_current_widget()
        self._draw_forecast_widget()
        self.draw_values(True)

    def _draw_current_widget(self):
        """draw a current tile"""
        self.lcd.background_color = self.colours['background_current']
        self.lcd.fill_rect(
            self.current_widget_pos[0],
            self.current_widget_pos[1],
            self.current_widget_pos[0] + 115,
            self.current_widget_pos[1] + 103
        )
        self.lcd.color = self.colours['border']
        self.lcd.draw_rect(
            self.current_widget_pos[0],
            self.current_widget_pos[1],
            self.current_widget_pos[0] + 115,
            self.current_widget_pos[1] + 103
        )

    def _draw_forecast_widget(self):
        """draw a forecast tile"""
        self.lcd.background_color = self.colours['background_forecast']
        self.lcd.fill_rect(
            self.forecast_widget_pos[0],
            self.forecast_widget_pos[1],
            self.forecast_widget_pos[0] + 115,
            self.forecast_widget_pos[1] + 103
        )
        self.lcd.color = self.colours['border']
        self.lcd.draw_rect(
            self.forecast_widget_pos[0],
            self.forecast_widget_pos[1],
            self.forecast_widget_pos[0] + 115,
            self.forecast_widget_pos[1] + 103
        )

Before we add values, we will improve our lives by adding a draw_number function to the Widget class. Almost all widgets are displaying some numbers so it is a waste of space to have a duplicated code. One function for all πŸ™‚

   def draw_number(self, pos_x, pos_y, font, current, previous, spacing=30):
        """draw a number"""
        self.lcd.transparency_color = font.get_transparency()
        for idx in range(0, len(current)):
            if previous is None or current[idx] != previous[idx]:
                self.lcd.draw_image(
                    pos_x + (idx * spacing),
                    pos_y,
                    font.get(int(current[idx]))
                )

and another helper, this time only for Openweather to get formatted data by key:

    def _get_value_current_weather(self, key, value, precision=2):
        """return None or rounded value from current readings"""
        return None if self.current_weather[key] is None else str(round(self.current_weather[key][value])).rjust(precision, '0')

    def _get_value_forecast_weather(self, key, value, precision=2):
        """return None or rounded value from current readings"""
        return None if self.forecast_weather[key] is None else str(round(self.forecast_weather[key][value])).rjust(precision, '0')

Ok, all helpers in place, back to value displaying: temperature, wind speed and pressure:

   def _draw_current_values(self, force=False):
        """draw current values"""
        current = self._get_value_current_weather('current', 'temperature_current')
        previous = self._get_value_current_weather('previous', 'temperature_current')
        if force or previous is None or current != previous:
            self.draw_number(
                self.current_widget_pos[0] + 50,
                self.current_widget_pos[1] + 5,
                self.fonts['15x28'],
                current,
                previous,
                20
            )

        current = self._get_value_current_weather('current', 'wind_speed')
        previous = self._get_value_current_weather('previous', 'wind_speed')
        if force or previous is None or current != previous:
            self.draw_number(
                self.current_widget_pos[0] + 50,
                self.current_widget_pos[1] + 39,
                self.fonts['15x28'],
                current,
                previous,
                20
            )

        current = self._get_value_current_weather('current', 'pressure', 4)
        previous = self._get_value_current_weather('previous', 'pressure', 4)
        if force or previous is None or current != previous:
            self.draw_number(
                self.current_widget_pos[0] + 30,
                self.current_widget_pos[1] + 72,
                self.fonts['15x28'],
                current,
                previous,
                20
            )

Before we see how it looks we need to enable this widget in main.py:

FONTS = {
    '24x42': numbers_24x42.Numbers(),
    '15x28': numbers_15x28.Numbers()
}
WIDGETS = {
    'openweather': OpenweatherWidget(
        ((0, 108), (125, 108)),
        lcd_tft,
        FONTS
    )
}
for sensor in WIDGETS:
    WIDGETS[sensor].draw_widget()

And once again, it looks better than on the picture:

And a big question, what is what? We need something to describe what value means. An icon should be perfect. But maybe we will do some upgrade in case of wind? We can add an icon with wind direction. We have information in form of a degree so we just need a square image with an arrow and we will use PIL to rotate it.

Let’s add image assets in init:

        self.icon = {
            'temperature': Image.open('assets/image/thermometer.png'),
            'compass': Image.open('assets/image/compass.png')
        }

and we can draw a thermometer icon in tile drawing:

self.lcd.transparency_color = (0, 0, 0)
self.lcd.draw_image(self.current_widget_pos[0] + 95, self.current_widget_pos[1] + 7, self.icon['temperature'])

and wind direction in value drawing:

        current = self._degree_to_direction(self.current_weather['current']['wind_deg'])
        previous = None if self.current_weather['previous'] is None else self._degree_to_direction(self.current_weather['previous']['wind_deg'])
        if force or previous is None or current != previous:
            self.lcd.transparency_color = ((255, 255, 255), (0, 0, 0))
            self.lcd.draw_image(
                self.current_widget_pos[0] + 92,
                self.current_widget_pos[1] + 44,
                self.icon['compass'].rotate(-1 * self.current_weather['current']['wind_deg'])
            )

We can also add thermometer icon to Node One widget.
There is one more thing that we need to correct, the background is a way to light and numbers are not clearly visible.
Now it looks very good:

Forecast weather tile is similar to current weather tile so let’s refactor and display both:

    def draw_widget(self):
        """draw a tiles"""
        self._draw_widget('current', self.current_widget_pos[0], self.current_widget_pos[1])
        self._draw_widget('forecast', self.forecast_widget_pos[0], self.forecast_widget_pos[1])
        self.draw_values(True)

    def _draw_widget(self, widget_type, pos_x, pos_y):
        """draw a tile"""
        self.lcd.background_color = self.colours['background_'+widget_type]
        self.lcd.fill_rect(pos_x, pos_y, pos_x + 115, pos_y + 103)
        self.lcd.transparency_color = (0, 0, 0)
        self.lcd.draw_image(pos_x + 95, pos_y + 7, self.icon['temperature'])
        self.lcd.color = self.colours['border']
        self.lcd.draw_rect(pos_x, pos_y, pos_x + 115, pos_y + 103)

And time to refactor data drawing. And ups:) Forecast differ in data fields, nothing a good if can’t fix πŸ˜€
The forecast has no temperature_current field instead is has temperature_max and temperature_min. So what to display? Let’s go with max.

And the result of our work:

Worker

We have our mocks and they are so lifeless 😦 We need to feed them with data. I was thinking how to connect the openweather service with the widget? We need to periodically refresh data in the widget so threads come in handy. But imagine that we have 10 workers and each has its own thread… blah.
Maybe we should write a threaded worker handler (nice name :P). It will aggregate all workers and execute them periodically. This, of course, requires us to write an abstract for Worker class to force an execute command implementation. This handler could work in such way that main thread executes all workers, sleeps for 1s and repeat. And we can add an option to set worker’s frequency.
This will allow us to set different frequency for each worker. Hopefully, we won’t ever need a perfect timing πŸ™‚
Worker abstract is small:

"""An abstract for Worker class"""
import abc


class Worker(metaclass=abc.ABCMeta):
    """Widget abstract"""
    @abc.abstractmethod
    def execute(self):
        """call a worker"""
        pass

    @abc.abstractmethod
    def start(self):
        """call a worker"""
        pass

    @abc.abstractmethod
    def shutdown(self):
        """stops worker and all services"""
        pass

What are start and shutdown function doing there? I thought that a worker should have them πŸ˜€
Quick update to the openweather worker:

    def execute(self):
        """executes worker"""
        print(self.weather())
        print(self.forecast())

Later, we would have to connect this to the widget.

Handler

Handlers, handlers everywhere πŸ™‚ To track time of execution we need a variable and a function. We can try and encapsulate worker in Job class:

class Job(object):
    """Class Job"""
    def __init__(self, worker, frequency):
        self.worker = worker
        self.worker.start()
        self.frequency = frequency
        self.tick = 0

    def can_execute(self):
        """checks if it is time to execute worker"""
        self.tick += 1
        if self.tick == self.frequency:
            self.tick = 0
            return True
        return False

    def execute(self):
        """executes a worker"""
        self.worker.execute()

    def shutdown(self):
        """shutdown worker"""
        self.worker.shutdown()

When we call it, start on the worker is called and a timer is initialized. It has a function to handle ticks and two proxies to worker class.
We are ready to move on, let’s create a workers handler:

class Handler(threading.Thread):
    """Handler class"""
    def __init__(self):
        threading.Thread.__init__(self)
        self.workers = {}
        self.tick = 1
        self.work = True

    def add(self, name, worker, frequency):
        """add worker to pool"""
        self.workers[name] = Job(worker, frequency)

    def run(self):
        """main loop"""
        while self.work:
            for worker in self.workers:
                if self.workers[worker].can_execute():
                    self.workers[worker].execute()
            time.sleep(1)

    def stop(self):
        """stops a thread"""
        self.work = False
        for worker in self.workers:
            self.workers[worker].shutdown()

Class Job is used directly from it so we won’t have to.
How to use it all?

workerHandler = WorkerHandler()
workerHandler.add('openweather', OpenweatherWorker('apikey'), 5)
workerHandler.start()

Job class calls start on worker so we have less code here πŸ™‚

Worker and widget

Finally, we can bind them together. How? We will pass widget to the worker. Simple and effective.
In worker, we need to change the execute function to:

    def execute(self):
        """executes worker"""
        self.widget.change_values({
            'current': self.weather(),
            'forecast': self.forecast()
        })

in widget function change_value to:

    def change_values(self, values):
        if 'current' in values:
            self.current_weather['previous'] = self.current_weather['current']
            self.current_weather['current'] = values['current']

        if 'forecast' in values:
            self.forecast_weather['previous'] = self.forecast_weather['current']
            self.forecast_weather['current'] = values['forecast']

and in main:

workerHandler.add('openweather', OpenweatherWorker('apikey', WIDGETS['openweather']), 5)

What have we here?

A double wind arrows :/ Yeh, it is a bug. We draw new value over old one, fix is simple, repaint area with background colour before an arrow is drawn.
Perfect.

Summary

We made it! The second widget added to Control Node, a weather widget. It displays data gathered from the Openweathermap site. It has two tiles, with current weather and the forecast. But there is still lots to do. It shows us only four data, rest is waiting for part three πŸ™‚

What else? We sugarcoated a GUI by adding icons. They are nice and informative.

Advertisements

3 comments

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s