Iran : Yazd

We took our first bus trip to reach Yazd from Shiraz using the Hamsafar company. Booking a bus trip is as easy as it is cheap in Iran so this is by far the best way to get around even tho it’s a bit slow mostly due to the police controls along the road.

Yazd was a shock as it’s a small and beautiful desert town with an unique athmosphere. This city still haunts me and remains my favorite of the trip. The baazar, the covered streets and the mud walls gives you a feeling which is difficult to describe.

We stayed at the very pleasant Orient Hotel and spent one night in a caravanserai where I tried my luck and succeeded to rent a motorcycle for one day ! That was a fun and incredible experience and we’ll always remember the look on the amuzed face of the Iranian people when they realized some tourists where riding a motorcycle among them.

05130001

05090008

05130004

05110008

05130003

05110012

05130008

05090012

05110006

Yazd is of course not only about desert and features some beautiful and peaceful gardens.

05110001

05110002

Consul on Gentoo Linux

As a clustering and distributed architecture enthusiast, I’m naturally interested in software providing neat ways to coordinate any kind of state/configuration/you-name-it over a large number of machines.

My quest, as many of you I guess, were so far limited to tools like zookeeper (packaged on my overlay but with almost no echo) and doozerd (last commit nearly 6 months ago) which both cover some of the goals listed above with more or less flavors and elegance (sorry guys, JAVA is NOT elegant to me).

I recently heard about consul, a new attempt to solve some of those problems in an interesting way while providing some rich fuctionnalities so I went on giving it a try and naturally started packaging it so others can too.

WTF is consul ?

consul logo

Consul is a few months’ old project (and already available on Gentoo !) from the guys making Vagrant. I especially like its datacenter centric architecture, intuitive deployment and its DNS + HTTP API query mecanisms. This sounds promising so far !

This is a descripion taken from the Hashicorp’s blog :

Consul is a solution for service discovery and configuration. Consul is completely distributed, highly available, and scales to thousands of nodes and services across multiple datacenters.

Some concrete problems Consul solves: finding the services applications need (database, queue, mail server, etc.), configuring services with key/value information such as enabling maintenance mode for a web application, and health checking services so that unhealthy services aren’t used. These are just a handful of important problems Consul addresses.

Consul solves the problem of service discovery and configuration. Built on top of a foundation of rigorous academic research, Consul keeps your data safe and works with the largest of infrastructures. Consul embraces modern practices and is friendly to existing DevOps tooling.

app-admin/consul ?

This is a RFC and interest call about the packaging and availability of consul for Gentoo Linux.

The latest version and live ebuilds are present in my overlay so if you are interested, please tell me (here, IRC, email, whatever) and I’ll consider adding it to the portage tree.

I want to test it !

Now that would be helpful to get some feedback about the usability of the current packaging. So far the ebuild features what I think should cover a lot of use cases :

  • full build from sources
  • customizable consul agent init script with reload, telemetry and graceful stop support
  • web UI built from sources and installation for easy deployment
# layman -a ultrabug
# emerge -av consul

Hope this interests some of you folks !

uWSGI v2.0.5.1

This release is important to me (and my company) as it officially introduces a few features we developed for our needs and then contributed to uWSGI.

Special congratulations to my co-worker @btall for his first contribution and for those nice features to the metrics subsystem with many thanks as usual to @unbit for reviewing and merging them so quickly.

new features

  • graceful reload of mule processes (Credits: Paul Egan) : SIGHUP is now sent to mules instead of directly killing them, by default you have 60 seconds to react before a SIGKILL
  • –metrics-no-cores, –stats-no-cores, –stats-no-metrics : don’t calculate and process all those core related metrics (gevent anyone ?)
  • reset_after_push for metrics (Credits: Babacar Tall) : this metric attribute ensures that the metric value is reset to 0 or its hardcoded initial_value every time the metric is pushed to some external system (like carbon, or statsd)
  • new metric_set_max and metric_set_min helpers (Credits: Babacar Tall) : can be used to avoid having to call “metric_get“ when you need a metric to be set at a maximal or minimal value. Another simple use case is to use the “avg“ collector to calculate an average between some *max* and *min* set metrics. Available in C and python.

See the full changelog here, especially some interesting bugfixes.

Iran : Shiraz

Our Iranian friend booked us a night train from Tehran to our next stop in the South of Iran : Shiraz.

The Tehran train station is well organized and as foreigners we had to show our passports to the station’s police where we were greeted with the big smile by the policemen. The train itself was good, clean, comfortable and on schedule even if it was quite slow : departure at 20h and we arrived at 11h30 a few kilometers up from Shiraz.

Travelling by train is always a great experience. As we woke up early in the morning, this is what was waiting for us behind the curtains :

05080012

Compared to Tehran, the streets are more typical and fitting what you’d expect from a desert city. That’s the exact kind of town you want to get lost in, just walking around and going into its nice and large bazaar.

05120002

05040005

05120005

05120006

Shiraz is known for its many palaces and beautiful gardens.

05120001

05120012

05090004

05090002

This city is also the usual starting point to visit the famous Persepolis. While the place is great, I must admit that we loved the abandonned park right next to it which is used as a playground and food camping by the schools and other Iranian tourists.

05120009

05120011

rsyslog v7.6.3

This version bump was long overdue sorry and it has happened only thanks to the great work of Thomas D. aka @Whissi, thanks again mate.

Please read carefully because this version introduces major ebuild changes, you’ll probably have to adapt your current configuration !

ebuild changes

"/var/log/syslog" log file is now deprecated

   Beginning with rsyslog-7.6, the "/var/log/syslog" log file will no
   longer being written per default. We are considering this file as
   deprecated/obsolet for the typical user/system.
   The content from this log file is still availble through other
   (dedicated) log files, see

     - /var/log/cron.log
     - /var/log/daemon.log
     - /var/log/mail.log
     - /var/log/messages

   If you really need the old "/var/log/syslog" log file, all you have to
   do is uncommenting the corresponding configuration directive in
   "/etc/rsyslog.d/50-default.conf".

   If you do so, don't forget to re-enable log rotation in
   "/etc/logrotate.d/rsyslog", too.
  • An additional input socket in /var/empty/dev/log (default chroot
    location) will be created per default
  • brand new and modern init script

rsyslog-7.6.3

Coming from the rsyslog release announcement page, this is what happened with the 7.6 branch release :

With 7.6 being the successor of the 7.5 development branch, everything that has been added there has now found its way into the stable version.

The major additions consist of :
- imrelp/omrelp now support TLS & (zip) compression
- impstats is now emitting resource usage counters, can directly emit delta values and can now be bound to a ruleset
- mmpstrucdata is a new module to parse RFC5424 structured data into JSON message properties
- mmutf8fix is a new module to fix invalid UTF-8 sequences
- mmsequence is a new module that helps with action load balancing
- new defaults for main/ruleset queues to be more enterprise-like

Also the new stable version has undergone a lot of bug fixes, performance improvements and optimizations that make rsyslog 7.6 a lot more reliable and performing than before.

Iran : Tehran

For our latest vacations, we spent two weeks in Iran and I must say that this country is astonishingly beautiful ! So forget what you think to know about this country as it’s either biased or doesn’t apply to its people who are very friendly and welcoming.

We started with two days in Tehran, a gigantic and lovely city where an Iranian friend we met on the internet invited us to a treck in the surroundings montains.

The first thing to have to get acquainted with is how to cross the streets. Seriously the traffic is madness and uncontrolled (no traffic lights) so you have to watch out everywhere. Then you discover their nice and cooled subway which is clean and shiny and so cheap you try and count four times as you’re sure you’ve heard the man wrong : 10 000 rials a ticket, yeah that’s like 0,2€…

06340006

06340008

Treck startpoint was Darban and then up to close to 3000m, thanks again Shayan. When we were awed by the beautiful mountains they had at the doorstep or their city, our Iranian friends told us that these mountains are also the cause of the massive pollution of Tehran in sping and summer because it blocks the winds from clearing the air…

05080008

05080011

05080010

Sunset from the Jamshidieh park, the tall tower on the background is the Milad Tower.

05060012

mongoDB v2.6.1

This is a great pleasure to announce the version bump of mongoDB to the brand new v2.6 stable branch !

This bump is not trivial and comes with a lot of changes, please read carefully as you will have to modify your mongodb configuration files !

ebuild changes

As a long time request and to be more in line with upstream’s recommendations (and systemd support) I moved the configuration of the mongoDB daemons to /etc so make sure to adapt to the new YAML format.

  • the mongodb configuration moved from /etc/conf.d/mongodb to the new YAML formatted /etc/mongodb.conf
  • the mongos configuration moved from /etc/conf.d/mongos to the new YAML formatted /etc/mongos.conf
  • the MMS agent configuration file has moved to /etc/mms-agent.conf

The init scripts also have been taken care of :

  • new and modern mongodb, mongos and mms-agent init scripts
  • their /etc/conf.d/ configuration files are only used to modify the init script’s behavior

highlights

The changelog is long and the goal of this post is not to give you an already well covered topic on the release notes but here are my favorite features :

  • MongoDB preserves the order of the document fields following write operations.
  • A new write protocol integrates write operations with write concerns. The protocol also provides improved support for bulk operations.
  • MongoDB can now use index intersection to fulfill queries supported by more than one index.
  • Index Filters to limit which indexes can become the winning plan for a query.
  • Background index build allowed on secondaries.
  • New cleanupOrphaned command to remove orphaned documents from a shard.
  • usePowerOf2Sizes is now the default allocation strategy for all new collections.
  • Removed upward limit of 20 000 connections for the maxIncomingConnections for mongod and mongos.
  • New cursor.maxTimeMS() and corresponding maxTimeMS option for commands to specify a time limit.

Make sure you follow the official upgrade plan to upgrade from a previous version, this release is not a simple drop-in replacement.

thanks

Special thanks go to Johan Bergström for his continuous efforts and responsiveness as well as Mike Limansky and Jason A. Donenfeld.

uWSGI v2.0.4

Quick post for an interesting version bump of uWSGI which brings an experimental loopengine for python3.4 asyncio (aka tulip) !

If you want to try it out, I added a python_asyncio USE flag. I’ve also made some cleanups on the ebuild wrt python versions and dropped older versions of uWSGI.

highlights

  • experimental asyncio loop engine (python 3.4 only)
  • httprouter advanced timeout management
  • purge LRU cache (v2) feature
  • allow duplicate headers in http parsers
  • faster on_demand Emperor management
  • fixed segfault for unnamed loggers

See the full changelog here.

After vacation bug hunting

Two weeks vacations always seem short yet the 900+ mails waiting for sorting on my Gentoo Linux inbox was a reminder that our beloved distribution is well alive ! So I guess it was time for a little bug killing spree :)

rabbitMQ v3.3.0

This release improves performance in a variety of conditions, adds monitoring information to identify performance bottlenecks, adds dynamically manageable shovels, and allows Java-based clients to reconnect automatically after network failure.

This release also corrects a number of defects in the broker and plugins, as well as introducing a host of smaller features as you can see on the changelog. Be warned that the behavior of the guest user has been altered !

I also fixed a long awaiting bug to bump the rabbitMQ C client to v0.5.0

redis v2.8.9

Johan Bergström is as always doing a great and helpful job and is actively working on redis, thanks mate !

  • [NEW] The HyperLogLog data structure. You can read more about it  in this blog post
  • [NEW] The Sorted Set data type has now support for lexicographic range queries, check the new commands ZRANGEBYLEX, ZLEXCOUNT and ZREMRANGEBYLEX, which are documented at http://redis.io

py3status v1.5

  • fixes installation via pip
  • added a –version command line argument to get the currently installed version of py3status

upcoming bumps

You might be interested in what’s next on the todo list :

  • With the help of Thomas D. aka @Whissi, we’re working on bumping and enhancing rsyslog to v7.6.3. For this a series of its dependencies have been bumped today as well.
  • mongoDB v2.6.0 is also on track, as usual the guys @mongodb have broken the scons building so it’s taking more time than it should to fix this hell (all help appreciated).

py3status v1.4

I’m glad to announce the release of py3status-1.4 which I’d like to dedicate to @guiniol who provided valuable debugging (a whole Arch VM) to help me solve the problem he was facing (see changelog).

I’m gathering wish lists an have some (I hope) cool ideas for the next v1.5 release, feel free to post your most adventurous dreams !

changelog

  • new ordering mechanism with verbose logging on debug mode. fixes rare cases where the modules methods were not always loaded in the same order and caused inconsistent ordering between reloads. thx to @guiniol for reporting/debugging and @IotaSpencer and @tasse for testing.
  • debug: dont catch print() on debug mode
  • debug: add position requested by modules
  • Add new module ns_checker.py, by @nawadanp
  • move README to markdown, change ordering
  • update the README with the new options from –help

contributors

Special thanks to this release’s contributors !

  • @nawadanp
  • @guiniol
  • @IotaSpencer
  • @tasse