One month in Turkey

Our latest roadtrip was as amazing as it was challenging because we decided that we’d spend an entire month in Turkey and use our own motorbike to get there from Paris.

Transportation

Our main idea was to spare ourselves from the long hours of road riding to Turkey so we decided from the start to use ferries to get there. Turns out that it’s pretty easy as you have to go through Italy and Greece before you set foot in Bodrum, Turkey.

  • Paris -> Nice : train
  • Nice -> Parma (IT) -> Ancona : road, (~7h drive)
  • Ancona -> Patras (GR) : ferry (21h)
  • Patras -> Piraeus (Athens) : road (~4h drive, constructions)
  • Piraeus -> Kos : ferry (~11h by night)
  • Kos -> Bodrum (TR) : ferry (1h)

Turkish customs are very friendly and polite, it’s really easy to get in with your own vehicle.

Tribute to the Nightster

This roadtrip added 6000 kms to our brave and astonishing Harley-Davidson Nightster. We encountered no problem at all with the bike even though we clearly didn’t go easy on her. We rode on gravels, dirt and mud without her complaining, not to mention the weight of our luggages and the passengers ;)

That’s why this post will be dedicated to our bike and I’ll share some of the photos I took of it during the trip. The real photos will come in some other posts.

A quick photo tour

I can’t describe well enough the pleasure and freedom feeling you get when travelling in motorbike so I hope those first photos will give you an idea.

I have to admit that it’s really impressive to leave your bike alone between the numerous trucks parking, loading/unloading their stuff a few centimeters from it.

IMG_20140905_130004

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

We arrived in Piraeus easily, time to buy tickets for the next boat to Kos.

IMG_20140906_164101  IMG_20140906_191845

 

 

 

 

 

 

Kos is quite a big island that you can discover best by … riding around !

IMG_20140907_121148

After Bodrum, where we only spent the night, you quickly discover the true nature of Turkish roads and scenery. Animals are everywhere and sometimes on the road such as those donkeys below.

IMG_20140909_180844

 

 

 

 

 

 

 

This is a view from the Bozburun bay. Two photos for two bike layouts : beach version and fully loaded version ;)

IMG_20140909_191337 IMG_20140910_112858

 

 

 

 

 

 

 

On the way to Cappadocia, near Karapinar :

IMG_20140918_142943

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

The amazing landscapes of Cappadocia, after two weeks by the sea it felt cold up there.

IMG_20140920_140433 IMG_20140920_174936 IMG_20140921_130308

 

 

 

 

 

 

 

Our last picture from the bike next to the trail leading to our favorite and lonely “private” beach on the Datça peninsula.

IMG_20140925_182326

 

 

 

 

 

 

 

 

 

 

 

 

py3status v1.6

Back from holidays, this new version of py3status was due for a long time now as it features a lot of great contributions !

This version is dedicated to the amazing @ShadowPrince who contributed 6 new modules :)

Changelog

  • core : rename the ‘examples’ folder to ‘modules’
  • core : Fix include_paths default wrt issue #38, by Frank Haun
  • new vnstat module, by Vasiliy Horbachenko
  • new net_rate module, alternative module for tracking network rate, by Vasiliy Horbachenko
  • new scratchpad-counter module and window-title module for displaying current windows title, by Vasiliy Horbachenko
  • new keyboard-layout module, by Vasiliy Horbachenko
  • new mpd_status module, by Vasiliy Horbachenko
  • new clementine module displaying the current “artist – title” playing in Clementine, by François LASSERRE
  • module clementine.py: Make python3 compatible, by Frank Haun
  • add optional CPU temperature to the sysdata module, by Rayeshman

Contributors

Huge thanks to this release’s contributors :

  • @ChoiZ
  • @fhaun
  • @rayeshman
  • @ShadowPrince

What’s next ?

The next 1.7 release of py3status will bring a neat and cool feature which I’m sure you’ll love, stay tuned !

Using uWSGI and Consul to design a distributed application

Foreword

Let’s say we have to design an application that should span across multiple datacenters while being able to scale as easily as firing up a new vm/container without the need to update any kind of configuration.

Facing this kind of challenge is exciting and requires us to address a few key scaffolding points before actually starting to code something :

  • having a robust and yet versatile application container to run our application
  • having a datacenter aware, fault detecting and service discovery service

Seeing the title of this article, the two components I’ll demonstrate are obviously uWSGI and Consul which can now work together thanks to the uwsgi-consul plugin.

While this article example is written in python, you can benefit from the same features in all the languages supported by uWSGI which includes go, ruby, perl ad php !

Our first service discovering application

The application will demonstrate how simple it is for a client to discover all the available servers running a specific service on a given port. The best part is that the services will be registered and deregistered automatically by uWSGI as they’re loaded and unloaded.

The demo application logic is as follows :

  1. uWSGI will load two server applications which are each responsible for providing the specified service on the given port
  2. uWSGI will automatically register the configured service into Consul
  3. uWSGI will also automatically register a health check for the configured service into Consul so that Consul will also be able to detect any failure of the service
  4. Consul will then respond to any client requesting the list of the available servers (nodes) providing the specified service
  5. The client will query Consul for the service and get either an empty response (no server available / loaded) or the list of the available servers

Et voilà, the client can dynamically detect new/obsolete servers and start working !

Setting up uWSGI and its Consul plugin

On Gentoo Linux, you’ll just have to run the following commands to get started (other users refer to the uWSGI documentation or your distro’s package manager). The plugin will be built by hand as I’m still not sure how I’ll package the uWSGI external plugins…

$ sudo ACCEPT_KEYWORDS="~amd64" emerge uwsgi
$ cd /usr/lib/uwsgi/
$ sudo uwsgi --build-plugin https://github.com/unbit/uwsgi-consul
$ cd -

 

You’ll have installed the uwsgi-consul plugin which you should see here :

$ ls /usr/lib/uwsgi/consul_plugin.so
/usr/lib/uwsgi/consul_plugin.so

 

That’s all we need to have uWSGI working with Consul.

Setting up a Consul server

Gentoo users will need to add the ultrabug overlay (use layman) and then install consul (other users refer to the Consul documentation or your distro’s package manager).

$ sudo layman -a ultrabug
$ sudo ACCEPT_KEYWORDS="~amd64" USE="web" emerge consul

 

Running the server and its UI is also quite straightforward. For this example, we will run it directly from a dedicated terminal so you can also enjoy the logs and see what’s going on (Gentoo users have an init script and conf.d ready for them shall they wish to go further).

Open a new terminal and run :

$ consul agent -data-dir=/tmp/consul-agent -server -bootstrap -ui-dir=/var/lib/consul/ui -client=0.0.0.0

 

You’ll see consul running and waiting for work. You can already enjoy the web UI by pointing your browser to http://127.0.0.1:8500/ui/.

Running the application

To get this example running, we’ll use the uwsgi-consul-demo code that I prepared.

First of all we’ll need the consulate python library (available on pypi via pip). Gentoo users can just install it (also from the ultrabug overlay added before) :

$ sudo ACCEPT_KEYWORDS="~amd64" emerge consulate

 

Now let’s clone the demo repository and get into the project’s directory.

$ git clone git@github.com:ultrabug/uwsgi-consul-demo.git
$ cd uwsgi-consul-demo

 

First, we’ll run the client which should report that no server is available yet. We will keep this terminal open to see the client detecting in real time the appearance and disappearance of the servers as we start and stop uwsgi :

$ python client.py 
no consul-demo-server available
[...]
no consul-demo-server available

 

Open a new terminal and get inside the project’s directory. Let’s have uWSGI load the two servers and register them in Consul :

$ uwsgi --ini uwsgi-consul-demo.ini --ini uwsgi-consul-demo.ini:server1 --ini uwsgi-consul-demo.ini:server2
[...]
* server #1 is up on port 2001


* server #2 is up on port 2002

[consul] workers ready, let's register the service to the agent
[consul] service consul-demo-server registered succesfully
[consul] workers ready, let's register the service to the agent
[consul] service consul-demo-server registered succesfully

 

Now let’s check back our client terminal, hooray it has discovered the two servers on the host named drakar (that’s my local box) !

consul-demo-server found on node drakar (xx.xx.xx.xx) using port 2002
consul-demo-server found on node drakar (xx.xx.xx.xx) using port 2001

Expanding our application

Ok it works great on our local machine but we want to see how to add more servers to the fun and scale dynamically.

Let’s add another machine (named cheetah here) to the fun and have servers running there also while our client is still running on our local machine.

On cheetah :

  • install uWSGI as described earlier
  • install Consul as described earlier

Run a Consul agent (no need of a server) and tell him to work with your already running consul server on your box (drakar in my case) :

$ /usr/bin/consul agent -data-dir=/tmp/consul-agent -join drakar -ui-dir=/var/lib/consul/ui -client=0.0.0.0

The -join <your host or IP> is the important part.

 

Now run uWSGI so it starts and registers two new servers on cheetah :

$ uwsgi --ini uwsgi-consul-demo.ini --ini uwsgi-consul-demo.ini:server1 --ini uwsgi-consul-demo.ini:server2

 

And check the miracle on your client terminal still running on your local box, the new servers have appeared and will disappear if you stop uwsgi on the cheetah node :

consul-demo-server found on node drakar (xx.xx.xx.xx) using port 2001
consul-demo-server found on node drakar (xx.xx.xx.xx) using port 2002
consul-demo-server found on node cheetah (yy.yy.yy.yy) using port 2001
consul-demo-server found on node cheetah (yy.yy.yy.yy) using port 2002

Go mad

Check the source code, it’s so simple and efficient you’ll cry ;)

I hope this example has given you some insights and ideas for your current or future application designs !

HD Daymaker LED Headlamp

2014-08-12-064921_571x472_scrot

Short post to share my experience with the Harley-Davidson Daymaker LED Headlamp.

I came to buy it because I was not satisfied with the standard lamp fitted on my sportster and I guess whoever has to drive by night would feel that unpleasant feeling to not actually be able to properly see what’s going on in front of you.

The LED Headlamp is worth the few hundred bucks it costs if at least for the sake of your own life but furthermore for the incredible improvement from the standard lamp. Don’t hesitate a second just go for it and it’s dead simple to mount yourself !

See the difference (passing lights) :

IMG_20140804_215931

IMG_20140804_224657

Now I feel way safer to drive on unlitten roads.

Europython 2014

I had the chance to participate to europython 2014 as my company was sponsoring the event.

IMG_20140725_161445-1024x576

This was a great week where I got to meet some very interesting people and hear about some neat python use cases, libraries and new technologies so I thought I’d write a quick summary of my biased point of view.

ZeroMQ

I had the chance to meet Pieter Hintjens and participate in a 3 hours workshop on ZeroMQ. This was very interesting and refreshing as to go in more depth into this technology which I’ve been using in production for several years now.

Pieter is also quite a philosophical person and I strongly encourage you to listen to his keynote. I ended up pinging him in real life for an issue I’ve been waiting for bug correction on the libzmq and it got answered nicely.

uWSGI

Another big thing in our python stack is the uWSGI application container which I love and follow closely even if my lack of knowledge in C++ prevents me for going too deep in the source code… I got the chance to speak with Roberto De Ioris about the next 2.1 release and propose him two new features.

Thanks a lot for your consideration Roberto !

Trends

  • Not tested = broken !
  • Python is strong and very lively in the Big Data world
  • Asynchronous and distributed architectures get more and more traction and interest

Videos

All the talks videos are already online, you should check them out !

Iran : Yazd

We took our first bus trip to reach Yazd from Shiraz using the Hamsafar company. Booking a bus trip is as easy as it is cheap in Iran so this is by far the best way to get around even tho it’s a bit slow mostly due to the police controls along the road.

Yazd was a shock as it’s a small and beautiful desert town with an unique athmosphere. This city still haunts me and remains my favorite of the trip. The baazar, the covered streets and the mud walls gives you a feeling which is difficult to describe.

We stayed at the very pleasant Orient Hotel and spent one night in a caravanserai where I tried my luck and succeeded to rent a motorcycle for one day ! That was a fun and incredible experience and we’ll always remember the look on the amuzed face of the Iranian people when they realized some tourists where riding a motorcycle among them.

05130001

05090008

05130004

05110008

05130003

05110012

05130008

05090012

05110006

Yazd is of course not only about desert and features some beautiful and peaceful gardens.

05110001

05110002

Consul on Gentoo Linux

As a clustering and distributed architecture enthusiast, I’m naturally interested in software providing neat ways to coordinate any kind of state/configuration/you-name-it over a large number of machines.

My quest, as many of you I guess, were so far limited to tools like zookeeper (packaged on my overlay but with almost no echo) and doozerd (last commit nearly 6 months ago) which both cover some of the goals listed above with more or less flavors and elegance (sorry guys, JAVA is NOT elegant to me).

I recently heard about consul, a new attempt to solve some of those problems in an interesting way while providing some rich fuctionnalities so I went on giving it a try and naturally started packaging it so others can too.

WTF is consul ?

consul logo

Consul is a few months’ old project (and already available on Gentoo !) from the guys making Vagrant. I especially like its datacenter centric architecture, intuitive deployment and its DNS + HTTP API query mecanisms. This sounds promising so far !

This is a descripion taken from the Hashicorp’s blog :

Consul is a solution for service discovery and configuration. Consul is completely distributed, highly available, and scales to thousands of nodes and services across multiple datacenters.

Some concrete problems Consul solves: finding the services applications need (database, queue, mail server, etc.), configuring services with key/value information such as enabling maintenance mode for a web application, and health checking services so that unhealthy services aren’t used. These are just a handful of important problems Consul addresses.

Consul solves the problem of service discovery and configuration. Built on top of a foundation of rigorous academic research, Consul keeps your data safe and works with the largest of infrastructures. Consul embraces modern practices and is friendly to existing DevOps tooling.

app-admin/consul ?

This is a RFC and interest call about the packaging and availability of consul for Gentoo Linux.

The latest version and live ebuilds are present in my overlay so if you are interested, please tell me (here, IRC, email, whatever) and I’ll consider adding it to the portage tree.

I want to test it !

Now that would be helpful to get some feedback about the usability of the current packaging. So far the ebuild features what I think should cover a lot of use cases :

  • full build from sources
  • customizable consul agent init script with reload, telemetry and graceful stop support
  • web UI built from sources and installation for easy deployment
# layman -a ultrabug
# emerge -av consul

Hope this interests some of you folks !

uWSGI v2.0.5.1

This release is important to me (and my company) as it officially introduces a few features we developed for our needs and then contributed to uWSGI.

Special congratulations to my co-worker @btall for his first contribution and for those nice features to the metrics subsystem with many thanks as usual to @unbit for reviewing and merging them so quickly.

new features

  • graceful reload of mule processes (Credits: Paul Egan) : SIGHUP is now sent to mules instead of directly killing them, by default you have 60 seconds to react before a SIGKILL
  • –metrics-no-cores, –stats-no-cores, –stats-no-metrics : don’t calculate and process all those core related metrics (gevent anyone ?)
  • reset_after_push for metrics (Credits: Babacar Tall) : this metric attribute ensures that the metric value is reset to 0 or its hardcoded initial_value every time the metric is pushed to some external system (like carbon, or statsd)
  • new metric_set_max and metric_set_min helpers (Credits: Babacar Tall) : can be used to avoid having to call “metric_get“ when you need a metric to be set at a maximal or minimal value. Another simple use case is to use the “avg“ collector to calculate an average between some *max* and *min* set metrics. Available in C and python.

See the full changelog here, especially some interesting bugfixes.

Iran : Shiraz

Our Iranian friend booked us a night train from Tehran to our next stop in the South of Iran : Shiraz.

The Tehran train station is well organized and as foreigners we had to show our passports to the station’s police where we were greeted with the big smile by the policemen. The train itself was good, clean, comfortable and on schedule even if it was quite slow : departure at 20h and we arrived at 11h30 a few kilometers up from Shiraz.

Travelling by train is always a great experience. As we woke up early in the morning, this is what was waiting for us behind the curtains :

05080012

Compared to Tehran, the streets are more typical and fitting what you’d expect from a desert city. That’s the exact kind of town you want to get lost in, just walking around and going into its nice and large bazaar.

05120002

05040005

05120005

05120006

Shiraz is known for its many palaces and beautiful gardens.

05120001

05120012

05090004

05090002

This city is also the usual starting point to visit the famous Persepolis. While the place is great, I must admit that we loved the abandonned park right next to it which is used as a playground and food camping by the schools and other Iranian tourists.

05120009

05120011

rsyslog v7.6.3

This version bump was long overdue sorry and it has happened only thanks to the great work of Thomas D. aka @Whissi, thanks again mate.

Please read carefully because this version introduces major ebuild changes, you’ll probably have to adapt your current configuration !

ebuild changes

"/var/log/syslog" log file is now deprecated

   Beginning with rsyslog-7.6, the "/var/log/syslog" log file will no
   longer being written per default. We are considering this file as
   deprecated/obsolet for the typical user/system.
   The content from this log file is still availble through other
   (dedicated) log files, see

     - /var/log/cron.log
     - /var/log/daemon.log
     - /var/log/mail.log
     - /var/log/messages

   If you really need the old "/var/log/syslog" log file, all you have to
   do is uncommenting the corresponding configuration directive in
   "/etc/rsyslog.d/50-default.conf".

   If you do so, don't forget to re-enable log rotation in
   "/etc/logrotate.d/rsyslog", too.
  • An additional input socket in /var/empty/dev/log (default chroot
    location) will be created per default
  • brand new and modern init script

rsyslog-7.6.3

Coming from the rsyslog release announcement page, this is what happened with the 7.6 branch release :

With 7.6 being the successor of the 7.5 development branch, everything that has been added there has now found its way into the stable version.

The major additions consist of :
- imrelp/omrelp now support TLS & (zip) compression
- impstats is now emitting resource usage counters, can directly emit delta values and can now be bound to a ruleset
- mmpstrucdata is a new module to parse RFC5424 structured data into JSON message properties
- mmutf8fix is a new module to fix invalid UTF-8 sequences
- mmsequence is a new module that helps with action load balancing
- new defaults for main/ruleset queues to be more enterprise-like

Also the new stable version has undergone a lot of bug fixes, performance improvements and optimizations that make rsyslog 7.6 a lot more reliable and performing than before.