Event-sourcing benefit: migrating between storage types

Here’s an example of one benefit of Event Sourcing …

How much work would typically result from “hey, we need to change platform and store our data in a different database …”? Even using NHibernate and going from one relational database to another, you’d potentially be looking at a significant piece of effort.

Now imagine instead trying to migrate between fundamentally different storage engines such as from SQL Server to Amazon S3 or from Oracle to MongoDB?!

Well, EventSourcing not only makes it possible, it also makes it trivially simple too.

Read On →

MongoDB provider for Elmah

As a follow up to my Elmah Error Logging with MongoDB Official Driver post: I’ve created a github repo for it and also published it as a NuGet package. Please let me know if you have any troubles (first attempt with NuGet!).

Simple Service Bus / Message Queue with MongoDB

A service bus or message queue allow producers and subscribers to communicate asynchronously so that a system can handle disconnects, processes being stopped and started or enable peaks of demand to be handled beyond what the subscriber can immediately cope with. The queue acts as a buffer that the producer writes to and the subscriber reads from.

There are lots of implementations such as NServiceBus, MassTransit, Rhino Service Bus and the cloud-provided services such as Amazon’s Simple Queue Service and Window Azure’s AppFabric Service Bus. Some take a little time to get started with and the cloud ones can also rack up charges pretty quickly if you are doing too much polling.

Often, all that is needed is something fairly simple to buffer messages between processes and persist them. I’ve been making good use of MongoDB recently in conjunction with Jonathan Oliver’s EventStore library for a CQRS-based project so it seemed the obvious place to start – why not use MongoDB to store the queue?!

Read On →

Elmah Error Logging with MongoDB Official Driver

This basically takes the work that Pablo M. Cibraro did to use Elmah with the Samus CSharp Driver and converts it to work with the Official 10Gen CSharp Driver instead plus a few additional minor changes:

  1. A capped collection is still used but the maximum size (in bytes) and the document limit can now be set using the ‘maxDocuments’ and ‘maxSize’ parameters in the configuration. By default the limit is based on size only with 100mb allocated.

  2. The paged-results for the Elmah reporting page are sorted in descending order so the latest errors are shown first. This uses the $natural sort order of the capped collection.

  3. I’ve used the native MongoDB ObjectId for the error id which should be slightly faster that using a Guid and sorts better (also, if you were interested in saving a few bytes this stores the date and time too so could avoid saving it separately).

  4. Finally, I’ve use the convention of calling the collection ‘Elmah’ when there is no ApplicationName set and ‘Elmah-ApplicationName’ when it is.

Read On →

Running ElasticSearch as a Service on Windows 2008 x64

I think I first started using Apache Lucene for full-text indexing as part of NHibernate Search. At some point I decided I needed more control and did my own indexing using Lucene directly. Now, it seems the easiest approach is to make use of a packaged up search service and so I’ve been looking at ElasticSearch. So far, I’m very happy with it – it’s doing everything it say’s on the box and lets me offload all the full-text indexing and search functionality.

The only issue I’ve come across is trying to run it as a service on 64-bit Windows 7 or Windows 2008. While there is a service-wrapper available it just wasn’t working for me and I think the x64 platform may be part of that as there was only a elasticsearch-windows-x86-32.exe included, no elasticsearch-windows-x86-64.exe. This service wrapper seems to be based off a product that doesn’t appear to have a free community edition for 64-bit Windows.

So, I had a hunt around for ‘how to run a Java app as a Windows Service’ and came across the Apache Commons Daemon or ‘procrun‘. This worked so I thought I’d share it here in case anyone else is trying to do the same thing.

Read On →

Limit MongoDB memory use on Windows without Virtualization

I’ve seen the question of how to control MongoDB’s memory usage on Windows come up several times and the stock answer always seemed to be “you can’t – it uses memory-mapped files and if you want to limit resources you need to use some form of virtualization to do it (HyperV, VMWare, Virtuozzo etc…)”.

win2008r2

If you are using MongoDB on a dedicated server then you generally want it to use all the memory it can but if you want to use it on a server shared with other processes (e.g. an IIS website using MongoDB for storage, maybe with SQL Server as well) then you will want to put a cap on how much it uses to ensure memory is kept available for the other processes.

So is it possible if you are not on a virtualized environment? Yes (otherwise this would be a very short blog post!) and we’ll explore how …

Read On →

Error with Azure local development storage and table named 'event'

While working on an Azure event-sourcing provider for my CQRS framework I came across a really strange problem so I’m posting the details in-case anyone else comes across a similar issue so they can save wasting as much time on it as I did! Basically, the local development storage doesn’t seem to like you having a table called ‘event’ (I haven’t tested it on the live system).

Read On →

Pattern for efficiently paging database rows on a web page

It shouldn’t be difficult to read rows from a database and render them on a web page – it’s a pretty fundamental operation that most web apps have to do but a surprising number don’t get quite right, especially when the volume of data grows and it can’t all be loaded and rendered within a single request. I’ve recently come across several ‘enterprise applications’ that simply fail to get it right – one extreme example loaded all the database rows into session state and then used LINQ to get and display the pages from that (!) and another took the approach of simply not paging the results at all which meant huge pages that were incredibly slow to load and as the data grew beyond the simple developer-test dataset.

So, I’m going to try and explain the technique I use for paging through rows which I’ve found works efficiently and is easy to re-use. It works well for both traditional and AJAX requests and handles sorting and filtering requirements as well. This is based on ASP.NET MVC, jQuery and NHibernate + SQL Server but the principals will be the same for other technologies (I’ve used the same basic approach with MongoDB as the back-end data-store).

Read On →

Running MongoDb on Microsoft Windows Azure with CloudDrive

I’ve been playing around with the whole CQRS approach and think MongoDb works really well for the query side of things. I also figured it was time I tried Azure so I had a look round the web to see if there we’re instructions on how to run MongoDb on Microsoft’s Azure cloud. It turned out there were only a few mentions of it or a general approach that should work but no detailed instructions on how to do it. So, I figured I’d give it a go and for a total-Azure-newbie it didn’t turn out to be too difficult.

Read On →

Benefits of NoSQL (MongoDb) for the Query-side of CQRS

As you may know I’ve been researching CQRS and the benefits of using this approach to developing systems. My focus at the moment is on the Query side of things and for this I’ve been comparing a SQL Server / NHibernate solution with a NoSQL alternative using MongoDb. For this, I’ve been using a simple forum app that I’ve been working on with a database of around 4m posts and 200k topics. Read On →