Tag Archives: python

py3status v0.12

I’m glad to announce a new release of py3status ! I would like to thank @drahier for reporting an issue he found after suspending his computer. I took the opportunity to add a feature which will be helpful at work since we now have a local package installing some modules we share between colleagues (thx to @lujeni).

changelog

  • bugfix : don’t hang horribly when resuming from a suspend (was caused by an IOError exception which could occur when reading/writing to a suspending system).
  • feature : allow multiple -i include_path options to be passed and handle all the modules thus found.
  • feature : do not try to execute private and special methods on user-written Py3status’ classes.

py3status v0.9

First of all py3status is on pypi ! You can now install it with the simple and usual :

$ pip install py3status

This new version features my first pull request from @Fandekasp who kindly wrote a pomodoro module which helps this technique’s adepts by having a counter on their bar. I also fixed a few glitches on module injection and some documentation.

py3status v0.8

I went on a coding frenzy to implement most of the stuff I was not happy with py3status so far. Here comes py3status code name : San Francisco (more photos to come).
San Francisco

PEP8

I always had the habit of using tabulators to indent my code. @Lujeni pointed out that this is not a PEP8 recommended method and that we should start respecting more of it in the near future. Well, he’s right and I guess it was time to move on so I switched to using spaces and corrected a lot of other coding style stuff which got my code a score going from around -1/10 to around 9.5/10 on pylint !

Threaded modules’ execution

This was the major thing I was not happy with : when a user-written module was executed for injection, the time it took to get its response would cause py3status to stop updating the bar. This means that if you had a database call to make to get some stuff you need displayed on the bar and it took 10 seconds, py3status was sleeping for those 10 seconds to update the bar ! This behavior could cause some delays in the clock ticking for example.

I decided to offload all of the modules’ detection and execution to a thread to solve this problem. To be frank, this also helped to rationalize the code better as well. No more delays and a cleaner handling is what you get, stuff will start appending themselves whatever the time they take to execute !

Python3

It was about time the examples available on py3status would also work using python3.

py3status v0.7

Some cool bugfixes happened since v0.5 and py3status broke the 20 github stars, I hope people are enjoying it.

changelog

  • clear the user class cache when receiving SIGUSR1
  • specify default folder for user defined classes
  • fix time transformation thx to @Lujeni
  • add Pingdom checks latency example module
  • fix issue #2 reported by @Detegr which caused the clock to drift on some use cases

Python : writing a proper setup for your project

For an open-source project to be adopted, it must be easy to install and test it. Before publicly releasing py3status I thus had to figure out how to get my python project installed properly for every user who would be interested in it.

The common and most efficient way of writing a setup process in python is by using the setuptools package and writing your own setup.py file. Doing so is not hard at all as there are quite a bunch of examples you can start from but my challenge was that py3status is not a library you install and then import on your python code, instead it must be seen and used as an executable available for all users (something like /usr/bin/py3status). I’ll cover the steps I used to achieve this kind of installation.

setup.py basics

You can see the setup.py file just like any other python program you write where you import the functions you need from setuptools.

import os
from setuptools import find_packages, setup

Basically, a setup.py file is just a call to setup with a fair amount of parameters depending on the size and complexity of your project. Let’s see a basic usage with no real magic.

setup(
	name='py3status',
	version='0.5',
	url='https://github.com/ultrabug/py3status/wiki',
	download_url='https://github.com/ultrabug/py3status',
	license='BSD',
	author='Ultrabug',
	author_email='ultrabug@sikritdomain.com',
	description='py3status is an extensible i3status wrapper written in python',
	long_description='this is a very long description which im writing for example',
	platforms='any',
	)

As you can see those parameters are just fields describing your project but there are of course more parameters you can use to become more specific about it such as which other packages it depends from or special operations to be made upon installation.

classifiers

As with a literal description, you must categorize your project so that it will be correctly understood by automatic classifiers for example. The classifiers parameter is a list of those categories which you can find a list here.

	classifiers=[
		'License :: OSI Approved :: BSD License',
		'Operating System :: POSIX :: Linux',
		'Programming Language :: Python',
		'Programming Language :: Python :: 2.5',
		'Programming Language :: Python :: 2.6',
		'Programming Language :: Python :: 2.7',
		'Programming Language :: Python :: 3',
		'Programming Language :: Python :: 3.2',
		'Topic :: Software Development :: Libraries :: Python Modules',
		'Topic :: Desktop Environment :: Window Managers :: i3wm',
		],

getting an executable from your python program

As I explained earlier, py3status must be used as an executable available in the users’ PATH just like any other binary or commands on the system. I was thrilled to discover that achieving this is a piece of cake using setuptools, you just have to use the entry_points parameter and it will be taken care of for you.

entry_points={
		'console_scripts': [
			'py3status = py3status:main',
			]
		},

So here I’m asking setuptools to create a script which will execute py3status’ main function. It will generate a python program that just does that, call it py3status, place it in /usr/bin and make it executable. Et voilà ! An important thing to note is that it also works in Windows and that’s how you’ll get a .exe from your python code !

Learn more on this subject by reading the excellent documentation on how to get started with setuptools.

Packaging py3status

So I exchanged some mails with Michael Stapelberg of i3wm who rightly pointed out that my initial installation method of py3status was un-pythonic. I was not satisfied of using a bash setup either and I couldn’t imagine a better opportunity to learn how to write a proper setup.py for my project.

Thanks to my Gentoo Linux packager experience, I knew what I had to do, so a few searchs and tests later I’m glad to announce that py3status installation is standard ! I of course also packaged py3status for Gentoo Linux users : meet x11-misc/py3status on my overlay.

py3status being a real command and not a simple python module, I had to find the way to have setuptools taking care of this for me. I was happy to find out that this is pretty easy and that it works on both Linux & Windows, it’s awesome !

I will explain all this in one of my next blog post as I’m sure it can be of interest.

Meet py3status

This is the first public release of one of my open-source projects, don’t hesitate to share some feedback and/or thoughts with me.

Background

As a sysadmin, I have a lot of consoles open on multiples desktops and my 30″ screen was still not enough to cover my needs. To make things short, I needed to spare every pixel I could and KDE was really frustrating me as it was wasting a lot of space and ran quite a bunch of useless stuff in the background (akonadi/nepomuk anyone ?).

Then came my cyclic rage about it and I finally found my precious : i3wm. I just love it as it is what I ever needed : a lightweight yet very functional and handy WM.

  • No more resizing my consoles to fit next to each other and I can still use floating windows for the needed applications.
  • No more huge and pixel-hungry task bar, just a simple and very efficient one.

customization

The problem when you start using something new and awesome is that you get a lot of ideas on what you could do with it and how you’d love to customize it. I mean, when using KDE or Gnome, your ideas are quickly shaped by the fact that you’d have to learn some exotic framework or language to implement them.

  • Did you ever ask yourself how to add your own stuff in your task bar on KDE or Gnome ?
  • What if the customization options you want are not available in your WM menus ?

Well, my answer was “never mind” tbh and I slowly even lost the idea of implementing anything on my task bar.

i3bar & i3status

After switching to i3wm, my first customization was to name my workspaces and setup my own colors to adjust the look & feel of my desktop. Then I started to tune the program responsible for displaying useful information on my bar : i3status. As you may know, you have some limited modules which can take care of displaying some useful information on your bar such as the free disk space on a disk partition or your wired/wireless network status.

But then I asked myself the same questions as I used to on my KDE days : what if I want more ? my own stuff on my task bar ?

Introducing py3status

Thanks to the i3bar open and simple protocol and the robust (even if somewhat limited) i3status program, I could finally hack into my bar. Naturally, I had to do it myself and there was a few examples available on the net but nothing really handy and extensible enough. That’s how I had the idea of developping py3status !

philosophy & goals

  • no extra configuration file needed
  • rely on i3status and its existing configuration as much as possible
  • be extensible, it must be easy for users to add their own stuff/output by writing a simple python class which will be loaded and executed dynamically
  • add some built-in enhancement/transformation of basic i3status modules output

available now on github

I’m glad to announce that I pushed it today on github ! You can start using py3status now and give your feedback. I hope this project will help users get more of their i3wm environment and encourage their hacking power !

Python : new zeroMQ and mongoDB drivers

pyzmq-2.2.0.1

This one is very interesting to me because the code from the mighty gevent-zeromq library which brought gevent support to pyzmq has been merged into it ! I find it very humble and positive for the Open Source community to see such merges and want to express my gratitude to Travis Cline and the zeroMQ team for that.

Migrating is as easy as :

# gevent-zeromq previous way
from gevent_zeromq import zmq

# pyzmq new way
from pyzmq.green import zmq

I strongly encourage you to read the changelog.

pymongo-2.4.2

Bugfix release, the main point is that PyMongo will no longer select a replica set member for read operations that is not in primary or secondary state. Here is the changelog.

My views on Python

We’re having a pretty hot debate in my company about the core development language we would like to embrace in order to enhance our work flow and unlock both our innovation and development iteration.

I thought of doing a blog post instead of writing an email summarizing my point of view about why we should choose python in case it could help other people to make their own mind or at least understand mine. I don’t want to be dragged into the typical “language x VS language y” type of post as there are a lot of those already but instead focus on specific use cases. Anyway, to be totally transparent with you, dear reader, I have to tell you that the main “opponent” we’re considering is Javascript / node.js.

Python is easy to learn and write

I work for an online marketing/advertising company and we have four different IT/development teams with specific work to do.  That means that we all have our own constraints to take into account and we use different technologies / languages to achieve them. Opening a debate on a development language rationalization thus means that people will have to be able to learn the one we will choose, the quicker the better.

Well let’s be honest, python is not widely taught on IT schools so our guys definitely WILL have to learn it. But python is one of the most simple language to learn as you can quickly test and make progress. This mainly comes from its coding syntax which makes the code cleaner and easy to read, also the language’s keywords are simple to remember, no boring brackets or semicolons to drive you crazy. Python is about coding clean and instinctively.

A simple and straightforward syntax is very important to me because :

  • it is easier to read
  • it is easier to maintain
  • your code is lighter
  • you spend less time coding so you spend more time thinking

Python everywhere

When choosing your core language, you must make sure you can rely on it today and tomorrow for a wide range of use cases. I find it very interesting to have a look at a list of its application domains as it does really show that with this language at our core, we would be able to achieve anything.

The versatility of python is my main argument in favor of its adoption. This is a mature and proven language with a lot of libraries and frameworks to suit all our present and future needs. It is widely supported on an extensive set of platforms and I can’t think of an open-source project not supporting python right from the start.

At this point, I will quote the folks at AppNexus from their conference in the recent PyData NYC 2012 :

"Python's versatility allows us to use it both for offline analytical tasks as well as production system development. Doing so allows us to bridge the gap between prototypes and production by relying on the same code libraries and frameworks for both, thereby tightening our innovation loop". 

They say python helped them grow very rapidly and efficiently by permitting them to focus on innovation, needless to say that I share their point of view and would love to see this happening in my company.

Python use cases in my company

Now let’s talk about our present and projected use cases. This is not an exhaustive list as I want to keep it simple and demonstrate the versatility I just talked about.

Scripting and automation

I am a sysadmin and I am lazy. Nothing new here okay, but that’s how I met python a bunch of years ago. At that time my company’s infrastructure became more complex everyday as was growing very rapidly. New servers were arriving and provided new functionalities and new technologies which lead more and more to heterogeneity in the things we had to monitor, automate and configure.

Python is a sysadmin’s heaven with all its libraries capable of handling complex tasks easily, even in our cluster environments where you have to deal with parallel and high availability computing. This is a big relief to know that whatever the task you’re asked to carry you can safely say : “python can do it”. The keyword here is efficiency.

Complete and complex applications

A lot of modern and cross-platform applications are written in python or based on it. I for one wrote an email parsing software a few years ago and it’s still kicking in production, its maintenance is easy and it has evolved smoothly with our growth and needs.

Another thing I like about this language is that it’s fast and can benefit from a semi-compiled “byte-code” which speeds up your application. No, python is not C++ and speed is not it’s biggest advantage of course but it’s really fast enough to compete with others easily.

Let’s sample some famous software written in python :

  • BitTorrent, original client, along with several derivatives
  • Dropbox, a web-based file hosting service
  • OpenStack, a cloud computing IaaS platform
  • Portage, the heart of Gentoo Linux
  • Ubuntu Software Center, a graphical package manager

Web applications

Python also have some solid and very powerful librairies able to manage asynchronous, real-time and scalable web applications and services. We already do have some of those robust web apps running in production and python demonstrates everyday all of the strenghts I already talked about here. We use librairies such as gevent along with web frameworks like flask and message queuing with zeromq. Someday I may write a post about our python web stack, it may be interesting to share about it.

I have been able to recode a web app written in .NET very quickly while enhancing it in every way possible. It is way faster, reliable, fault tolerant and maintainable that it was before. Thanks to python we have a short development iteration which proves itself everyday as the application grows and is capable to meet and achieve any new challenge we’re asked to take care of. I’m convinced that no other language could have been so powerful and versatile than python to do so.

We’re not the only ones thinking and experiencing this of course, still in the list we can see :

  • Google uses Python for many tasks including the backends of web apps such as Google Groups, Gmail, and Google Maps, as well as for some of its search-engine internals
  • AppNexus uses Python for some of their web apps backend
  • YouTube uses Python “to produce maintainable features in record times, with a minimum of developers”
  • Yahoo! Groups uses Python “to maintain its discussion groups”

That’s some big players indeed and it’s interesting to see they use python for their web app backends.

Conclusion

I am not a software developer as I never took strong development courses at school. I am a sysadmin of complex, clustered and heterogeneous environments so this affects my development standards and point of view in a way that my expectations will surely be different from a pure developer.

My main concerns can be defined with words like proven, easy, clean, versatile, maintainable, fast (to code and execute), scalable, fault-tolerant and cross-platform. All of my choices have been based on those standards and concerns and I think they apply well in our debate because I chose Python to meet them all and I’ve never been disappointed or limited by it.

I hope this post reflects my thoughts and helped you understand them. I will tell you about the result and the decision my company made on this debate.