Tag Archives: mongodb

mongoDB v2.4.7 & pymongo v2.6.3

First of all, I’d like to point out a quite big change in the Gentoo mongodb package. The Chromium team responsible for the v8 package decided to stop its maintenance as it was too much trouble to be used efficiently as a shared library (mainly due to upstream’s breakage behavior). Even tho I don’t like bundled libraries on sources, I understand my fellow developers point of view.

I’ve thus been asked and did switch the mongodb ebuild to use the bundled v8 library. This means that mongodb has no more v8 packaging dependency now. The mongodb v2.2.x users are advised that since upstream does not bundle the v8 lib in their source, I dropped the v8 USE flag and support altogether on this version (it’s not officially supported anyway) !

This being said, I’ll drop the old ebuilds from tree on the next releases iterations.

mongodb-2.4.7

Yet another bugfix release on this unfamous 2.4.x series :

  • Fixed over-aggressive caching of V8 Isolates
  • Removed extraneous initial count during mapReduce
  • Cache results of dbhash command
  • Fixed memory leak in aggregation

pymongo-2.6.3

BSON parser hardening and fixes in the connection pool mechanism. More info here.

 

mongoDB v2.4.6 & pymongo v2.6

mongodb-2.4.6

The folks at 10gen discovered a severe bug in the mongoDB chunk migration process on sharded environments.

Basically, depending of the size of your documents, there was a chance that some get lost during data migration ! Relax tho, this case affected only the chunks containing documents which size are in the range of 16,776,185 and 16,777,216 bytes (inclusive) so this means that if you don’t have quite big documents in your cluster, you have not been affected by this bug.

Still, as a maintainer and production user of mongoDB, this is not the kind of news I like to hear especially when thinking of you Gentoo users. On top of this, I had the bad surprise to experience again the stale replication bug that was supposed to be fixed on 2.4.5 on my production cluster.

So I decided this was time for a major cleanup of the mongoDB ebuilds in portage to make sure we’re not shipping known broken versions of mongoDB to you guys. I thus :

  • dropped all obsolete <mongodb-2.2 ebuilds (I warned about this quite some time ago now)
  • dropped the known bugged versions of mongodb (<2.2.6 and <2.4.6)
  • cleaned up all obsolete files in $FILESDIR
  • added, on the v2.4.6 ebuild, the embedded-v8 USE flag as a user convenience so you can now have packages requiring v8-3.19 and mongodb installed on your machine. I added an explicit warning about this as this is not the way to go on normal usage as this is against Gentoo policy.
*mongodb-2.4.6 (21 Aug 2013)
*mongodb-2.2.6 (21 Aug 2013)

  21 Aug 2013; Ultrabug <ultrabug@gentoo.org> -mongodb-2.0.7-r1.ebuild,
  -mongodb-2.0.7-r2.ebuild, -mongodb-2.0.8-r1.ebuild, -mongodb-2.0.8-r2.ebuild,
  -mongodb-2.2.0-r1.ebuild, -mongodb-2.2.0-r2.ebuild, -mongodb-2.2.4.ebuild,
  +mongodb-2.2.6.ebuild, -mongodb-2.4.5.ebuild, -mongodb-2.4.6_rc1.ebuild,
  +mongodb-2.4.6.ebuild, -files/mongodb-1.8.5-fix-smokepy.patch,
  -files/mongodb-1.8-fix-scons.patch, -files/mongodb-2.2-fix-scons.patch,
  -files/mongodb-2.2-fix-sconscript.patch,
  -files/mongodb-2.4.4-fix-sharedclient.patch, -files/mongodb.initd,
  -files/mongodb-linux3.patch, -files/mongos.initd, metadata.xml:
  version bump, add embedded-v8 USE, drop critically bugged versions, drop
  obsolete versions, filesdir cleanup

I understand some people may still need some of those ebuilds so if that’s the case, just shout and I’ll gladly add them to my overlay so you can still use them easily.

other highlights

  • Improved replication robustness in presence of high network latency
  • Resolved replica set initial sync issue on certain virtualized platforms
  • Resolved sharding migration issue that produced excessive small chunks
  • Resolved C++ client shutdown issues

pymongo-2.6

This one is quite interesting as it brings both new and improved features as well as some bug fixes. Also note that they fixed some gevent compatibility stuff.

highlights / explanations

  • The max_pool_size option actually means what it says now. Pymongo will open at most this number of sockets to your servers. Do remember that if you share a connection between threads, then your (max_pool_size+1) thread will wait for a socket to be freed before being able to process your command.
  • waitQueueMultiple and waitQueueTimeoutMS options will help you define how much and how long you want a process to wait for a socket to be available before raising an exception.
  • Pymongo automatically splits batch inserts into 48MB chunks so you don’t have to worry about pushing a huge list of documents for insertion.
  • Support for aggregation cursors (for use with dev version 2.5.1, not used on production now)
  • Support for exhaust cursors. When you queried a large amount of data, the client had to ask the server for each batch of results. An exhaust cursor will instead stream batches to the client as quick as possible. This make pulling large sets of data faster and more reliable than before !

You can see the full extend of this bump on the pymongo Jira.

mongoDB v2.4.5 & rabbitMQ v3.1.3

Quick catch-up on recent bumps I’ve made for rabbitMQ and mongoDB before posting about EuroPython2013 !

mongoDB v2.4.5

The 2.4 branch is definitely the most bugged one I’ve seen so far. We waited until 2.4.4 at work before migrating but this was not enough and we got hit for a few weeks by a bug which finally got fixed in 2.4.5. Also this release fixes two security bugs so you’re strongly advised to upgrade.

highlights

  • CVE-2013-4650 – Improperly grant user system privileges on databases other than “local”.
  • CVE-2013-4650 – Remotely triggered segmentation fault in Javascript engine.
  • config server performance improvements
  • improve initial sync resilience to network failure (that’s the one)

flask-pymongo v0.3.0

@dcrosta finally had the time to take care of my pull request for flask-pymongo. We now rely on pymongo’s MongoClient parameters’ validation instead of implementing them again on flask-pymongo and added connect and socket timeout options.

rabbitMQ v3.1.3

A roll-up bugfix release mainly.

highlights

  • fix startup failure when using SSL with Erlang/OTP R16B01
  • fix queue crash requeuing in-memory messages (since 2.7.0)
  • fix leak affecting HA/mirrored queues (since 3.0.0)
  • fix bug that lead to incorrect reporting of accumulated stats (since 3.1.2)

 

mongoDB : latest releases

mongodb-2.4.4

Just bumped it to portage and fixed an open bug along. This is yet another bugfix release which backports the switch to the Cyrus SASL2 library for sasl authentication (kerberos). Dependencies were adjusted so you no longer need libgsasl on your systems (remember to depclean).

highlights

  • config upgrade fails if collection missing “key” field
  • migrate to Cyrus SASL2 library for sasl authentication
  • rollback files missing after rollback

pymongo-2.5.2

This one is important to note and I strongly encourage you to upgrade asap as it fixes an important security bug (CVE-2013-2132). I’ve almost dropped all other versions from tree anyway…

highlights 2.5.x

  • support GSSAPI (kerberos) authentication
  • support for SSL certificate validation with hostname matching
  • support for delegated and role based authentication

mongodb-2.5.x dev

What’s cooking for the next 2.6 releases ? Let’s take a quick look as of today.

  • background indexing on secondaries (hell yes!)
  • new implementation of external sort
  • add support for building from source with particular C++11 compilers (will fix a gentoo bug reported quite a long time ago)
  • mongod automatically continues in progress index builds following restart

mongoDB and Pacemaker recent bumps

mongoDB 2.4.3

Yet another bugfix release, this new stable branch is surely one of the most quickly iterated I’ve ever seen. I guess we’ll wait a bit longer at work before migrating to 2.4.x.

pacemaker 1.1.10_rc1

This is the release of pacemaker we’ve been waiting for, fixing among other things, the ACL problem which was introduced in 1.1.9. Andrew and others are working hard to get a proper 1.1.10 out soon, thanks guys.

Meanwhile, we (gentoo cluster herd) have been contacted by @Psi-Jack who has offered his help to follow and keep some of our precious clustering packages up to date, I wish our work together will benefit everyone !

All of this is live on portage, enjoy.

mongoDB v2.4.2 released

After the security issue related bumps of the previous releases which happened last weeks it was about time 10gen released a 2.4.x fixing the following issues:

  • Fix for upgrading sharded clusters
  • TTL assertion on replica set secondaries
  • Several V8 memory leak and performance fixes
  • High volume connection crash

I guess everything listed above would have affected our cluster at work so I’m glad we’ve been patient on following-up this release 🙂 See the changelog for details.

mongoDB v2.4.1 and pymongo 2.5 released

10gen released a critical update for mongoDB 2.4.0 which affected queries on secondaries, you should upgrade asap. The python mongo driver followed the 2.4.x releases and got bumped to 2.5 this week-end. I am pleased to announce that I took the chance to add the kerberos authentication support to both ebuilds while bumping them.

pymongo-2.5

  • GSSAPI (Kerberos) authentication
  • SSL certificate validation with hostname matching
  • Delegated and role based authentication

mongoDB : v2.4.0 released

A few months ago, I pointed out what was coming with this release and did an update of this cooking 2.4.0 later. Yesterday, 10gen announced the release of the new stable branch of mongoDB v2.4.0. Instead of talking about it again, I’ll focus on what this release brings to Gentoo users as I’m glad to announce that it’s already available in portage.

SSL support

First of all, I think it was a good time to close bug #421289 and finally enable the SSL support via the ssl USE flag. I’ll support it as much as upstream does, so don’t expect some big magic about it.

Shared client library

Since this has always been a mess, I also added the sharedclient USE flag so that users who really need the client shared library can toggle its installation easily. This also permits me to isolate possible problems from the main ebuild.

Upgrading to 2.4

This is seamless unless you’re running a sharded cluster ! In this case, take great care of what you do and note that the upgrade is only possible if your cluster is running v2.2 ! Please read with care the upgrade plan.

mongoDB v2.2.3 & uwsgi v1.4.5

Today’s bumps are interesting, kudos to both upstreams for their work.

mongodb-2.2.3

The main benefit of this version is the performance improvements on replicaSets. You can see the changelog here.

uwsgi-1.4.5

This one is quite a bump. I was particulary interested to know that the memory leak discovered on the gevent loop got fixed. Have a look at all the nice backports too, they’re worth it.

  • added –python-version and –perl-version
  • fixed a gevent memory leak
  • backported –extract option
  • added HTTP_Authorization support in mod_proxy_uwsgi
  • backported –php-fallback
  • backported perl uwsgi::postfork and uwsgi::atexit
  • fixed a memory leak with –http-socket during uploads
  • fixed unix signals usage in mules and spoolers
  • reset cores/requests on worker startup
  • fixed a refcnt bug in python uwsgi.workers()
  • fixed async mode when multiple fds are in place
  • fixed a cache collision bug
  • backported –emperor-procname
  • backported –touch-reload refactoring from 1.5
  • backported master+emperor fix from 1.5
  • new rbtree implementation (based on nginx) backported from 1.5
  • backported new logvars from 1.5