CQRS - yes or no?

Coordinator
Nov 10, 2011 at 9:45 AM
Edited Nov 10, 2011 at 9:47 AM

In our meeting on Nov. 8, we mostly rejected the idea of using CQRS for the application, based on our inexperience with it and on the fear that it wouldn't allow us to create as many features on Nov. 26 as we'd like. We also said to use some sort of SQL Server. (See http://godfather.codeplex.com/wikipage?title=Architecture&referringTitle=Documentation.)

Now, Jörg has started checking in a base CQRS architecture (using RavenDB), so the initial setup is already done.

The question is: should we se CQRS and RavenDB for the app or would it be overkill (for an application with just 5 users), hampering our productivity?

Coordinator
Nov 10, 2011 at 10:34 AM

CQRS bedeutet, eine Auftrennung der Schreib- und Leseseite.

Ladet euch den aktuellen Quellcode herunter. Ich habe das Grundgerüst gebaut und ein Beispiel für das Registrieren eines Kindes hinzugefügt. Die Auftrennung bedeutet: wird eine Aktion wie etwa "Speichern"-Button gedrückt von der Oberfläche ausgeführt, dann wird ein Command mit den Werten aus den Textboxen aus der UI erstellt.

_bus.Send(new RegisterNewChild(Guid.NewGuid(), txt_Firstname.Text, txt_Lastname.Text, Convert.ToDateTime(txt_Birthday.Text)));

 

Das Beispiel habe ich von Sponsorship.Ui.Modules.Child.RegisterNewChildForm. Um die Leseseite zu aktualisieren wird ein Domain-Eventhandler verwendet. D.h. die Daten die im Command zur Verfügung gestellt wurden sind im Domain-Event. Ein Domain-Event ist eine einfache Klasse. Die bereits fertige Infrastruktur kümmert sich um das Weiterleiten der Daten und das der entsprechende Handler die gewünschten Daten bekommt. Der Einfachheit halber kopiere ich mal den gesamten Code der für die Leseseite relevant ist rein:

public class ChildListView : HandlesEvent<NewChildRegistered>
    {
        private readonly IDocumentStore _documentStore;

        public ChildListView(IDocumentStore documentStore)
        {
            _documentStore = documentStore;
        }

        public void Handle(NewChildRegistered domainEvent)
        {
            using (var session = _documentStore.OpenSession())
            {
                var dto = new ChildListDto { AggregateRootId = domainEvent.ChildId, Name = domainEvent.Firstname + " " + domainEvent.Lastname };
                session.Store(dto);
                session.SaveChanges();
            }
        }
    }

    public class ChildListDto : Dto
    {
        public string Name { get; set; }
    }
Als Backend wird im Moment RavenDB verwendet, weil keinerlei Mapping benötigt wird und RavenDB auf jedem Windows (auch Server) läuft. Kein NHibernate, kein Entity Framework, kein SQL wird benötigt.
Aber: ihr schreibt die Applikation am 26. - ich wollte eine Coding Übung machen und das Projekt hat sich angeboten.
Coordinator
Nov 10, 2011 at 10:46 AM

Vorgehen beim Sample für einen schnellen Überblick:

  1. in das Verzeichnis "Sponsorship\packages\RavenDB.1.0.499\server" wechseln und Raven.server.exe starten -> fertig, die DB läuft
  2. Visual Studio Solution "Sponsorship\Sponsorship.sln" öffnen
  3. Sponsorship.Ui.Form1 in Codeansicht öffnen und button1_Click Methode ansehen -> es wird ein Modal Dialog Fenster geöffnet um ein neues Kind zu registrieren
    //Show a submodule - in this example as a modal dialog
    ObjectFactory.GetInstance<RegisterNewChildForm>().ShowDialog();
    
  4. Sponsorship.Ui.Modules.Child.RegisterNewChildForm in Codeansicht öffnen und Methode btn_Save_Click() ansehen -> Command erstellen und auf den Bus legen (bedeutet im Endeffekt Domainevent werden in den Eventstore gescrhieben und Domain-Eventhandler werden aufgerufen)
  5. Sponsorship.ReadModel.ChildListView ansehen -> Leseseite: die Daten werden in RavenDB hinzugefügt

Vorteil von Event Soucing: ihr habt das Auditing kostenlos dabei. Bsp.: welche Überweisungen wurden vom Paten historisch gemacht. Plus: Auswertungen die noch nicht bekannt sind können im Nachhinein leicht erstellt werden -> Domain Events müssen nur nochmal abgespielt werden

Vorteil von RavenDB (oder generell dokumentenorientierte DBs): kein Mapping, schnellere Entwicklung, kein Problem bei concurrent users, kostenlos für Open Source Projekte...

Vorteil verwendete Eventstore Library von Jonathan Oliver: es gibt viele Adapter. ich habe jetzt mal die RavenDB Variante genommen. Es gibt aber auch Implementierungen (über NuGet zu beziehen) für SQL Server, MySQL, MongoDB,...

 

 

 

 

 

 

 

 

Coordinator
Nov 10, 2011 at 10:50 AM

Wen es interessiert von wo die beiden Klassen "RegisterNewChild" (=Command) und "NewChildRegistered" (=Domain Event) kommen, macht einen Blick in das T4 Template Contracts.tt im Projekt "Sponsorship.Domain". Im Moment ist es in Zeile 53 ;-)

Beim Speichern des T4 Templates "Contracts.tt" werden die Klassen neu generiert. Das Ergebnis sieht man in der Datei "Contracts.cs".

Coordinator
Nov 10, 2011 at 1:33 PM

Thanks Jörg!

I for my part like what I'm seeing :-) But I would also like to hear from others involved that are not so familiar with this architecture.

One question though:
Correct me if I'm wrong, but I think the current implementation can lead to inconsistencies in a multiuser environment. 
In our scenario it's not a likely case, but what happens if multiple users are working concurrently on the same aggregate (e.g. Sponsor).
Every users works with his own instance of EventStore but these instances all access one and the same RavenDB server in the network.

Is this case somehow handled?

Sebastian

Coordinator
Nov 10, 2011 at 1:55 PM

RavenDB should be started on a central server in production. Pretty the same you would do it with any relational database. The EventStore is also located on the same RavenDB Server (for now it uses the same connection string from the app.config).

>> Every users works with his own instance of EventStore but these instances all access one and the same RavenDB server in the network.

-> No, the events are therefor stored on the server. No RavenDB instance will be started on the client computer in production.

If users are working on the same aggregate and when they are sending the same command and it produces the SAME event then a concurrency exception is thrown. This should be handled in client code (f.e. show a message box).

Coordinator
Nov 10, 2011 at 2:30 PM

Yes, I get that there is only one Raven Server in the network, otherwise everybody would only work on his/her own data ;-)

I was not very clear about what I meant with "EventStore". I didn't mean the actual data store but the connection to access it. I think in JO's EventStore its called IEventStream.
So to be more precise: As I see it, every client creates/opens a connection to write new events on its own and as there is no Version property in your commands/events/readmodel I wonder how you can ensure that there was no data written since you read data the last time -> that the events you are trying to commit aren't based on already stale data.

Sebastian

Coordinator
Nov 10, 2011 at 6:26 PM

There are two options in Jonathans EventStore. You can send the version number with the command (so you have to store the version number with the dto's on the read side) or you don't send it. When you don't send it it will set it internally to the value "0".

Furthermore there's only a sample for registering a new child for now (look at Sponsorship.Domain.Child.ChildCommandHandler). So this will be version 1 of the new aggregate. What you will do in the command handler for commands that are not creating an aggregate (read it as "Insert" in SQL terms) but will call a method on an aggregate that is updating the internal state ("Update" in SQL terms) is to call

_repository.GetById<Child>(id);

If you have saved the version with the dto on the read side you will call it with the overload:

var child = _repository.GetById<Child>(id, version);

The EventStore will bring the aggregate to the latest state by replaying the events on the aggregate (the child in this example). In the next line call the method on the child aggregate root with the parameters from the command. Third line: save the aggregate by getting the new events from the aggregate and store it in the EventStore. It's always the same three lines:

public void Handle(ChangeBankAccountOfChild command)
        {
            //get the aggregate by replaying the stored events
            var child = _repository.GetById<Child>(command.Id);
            //execute domain logic - it raises domain events or throws an exception
            child.ChangeBankAccountOfChild(command.ChildId, command.BankAccount, command.BankName);

            //save new raised events of aggregate into eventstore
            _repository.Save(child, Guid.NewGuid());
        }
The difference:

when you are sending the command with a version number the EventStore makes sure that the max version number of the stored events of that aggregate is the same. Otherwise it will throw a concurrency exception.
when your are sending the command without a version number it will load the aggregate into the current (internal) state and executes the command. If the business logic fails (for example you have a child and it cannot be deactivated twice) than you will get a business exception.
It's a valid question if we should save the version number on the read side or it it's just an overhead. As fare as I know there should be five users in total.
Coordinator
Nov 11, 2011 at 6:46 AM

Well I would prefer using CQRS, for multiple reasons:

- the event store is something i really love (well it could also be added without cqrs) 

- no need for an ORM at the current setup

- people will learn more and since a lot of the base is already written it should be possible to finish in one day

 

Two points that need to be kept in mind:

- Martin needs to extend it so we should primary focus on his needs/wishes imo

- People who missed the CQRS talks in the past might slow us down since we might get a lot more questions about the code

 

As i stated above already as long as Martin is fine with it I would prefer using CQRS.

Coordinator
Nov 11, 2011 at 7:27 AM

What are the Pros / Cons of CQRS? IMHO there are 2 questions to evaluate it for this project:

1. Will it slow us down that we cannot fullfill the minimum requirements? 

2. Is this the best technology for this kind of application or is it over the top?

Coordinator
Nov 11, 2011 at 8:02 AM
Edited Nov 11, 2011 at 8:37 AM

There are, in principle, three ways to structure this application:

  1. make it a CQRS application (with event sourcing, since Jörg has set it up) with a  new and exciting architecture,
  2. make it a classic DDD-style application with a domain model mapped against a database (e.g., using NHibernate or Entity Framework),
  3. make it a CRUD-style application that (more or less) directly binds UI controls against the database.

In my opinion, fun-wise and regarding the learning effect, option 1 would be the way to go. But even though it is the most stupid (and least interesting) approach, option 3 would probably be the fastest for this limited set of use cases. Option 2 is somewhere in the middle.

To answer Martin's questions:

  • I think CQRS will slow us down unless everyone comes to the session on Nov. 26 with a thorough understanding of the existing code and architectural approach.
  • It is over the top for (exactly) this kind of application, but it might get more people to work on the project.

I don't really care much whether we choose 1, 2, or 3 - in fact, I'd gladly let Martin decide. But I think that if we choose CQRS, everyone participating must read into the source code, must understand the architecture, and must know how to add new use cases. Otherwise, we won't get the application done.

Additional important point: Jörg won't be with us on Nov. 26. Is there anyone else who has already implemented a project with CQRS and could answer tricky questions if they arise?

Coordinator
Nov 11, 2011 at 8:50 AM

Thanks Fabian to sum it up. I think you have hit the nail. It is a valid question if CQRS is the right architecture for this application with less business logic. I see the advantage in the EventStore because you need some kind of auditing within the payment aggregate anyway.

In my opinion it is more difficult to learn somebody NHibernate, Entity Framework or other O/R Mappers than to describe the concept of CQRS and Eventsourcing with samples. It should fit on one paper. I think the learn effect is bigger with the CQRS way and therefor the will to add some use cases after the 26th will be higher. To be honest I'm not that interested in helping to maintain a CRUD app but I'm really not dissapointed if you choose to take that way.  I will call Martin directly.

Coordinator
Nov 11, 2011 at 10:57 AM

I have some experience - not from a production project though, but nevertheless - and I think I could answer most questions that will arise.

As Fabian said, I think it is very important that everybody who wants to help, should be familiar with this architecture and the current codebase before the 26th.
It's true that the concepts of CQRS are not that complicated, but it would still require some time to teach them basic things while they could start a classical CRUD / EF app right away.
Don't get me wrong. I'm still a fan of the idea to start a CQRS project - this could be of great help for other developers who are looking desperately for sample implementations of the architecture.

So thanks again Jörg for your effort. BTW I really like the way you implemented the bootstrapping process.

Coordinator
Nov 11, 2011 at 2:28 PM
public void Handle(ChangeBankAccountOfChild command)
        {
            //get the aggregate by replaying the stored events
            var child = _repository.GetById<Child>(command.Id);
            //execute domain logic - it raises domain events or throws an exception
            child.ChangeBankAccountOfChild(command.ChildId, command.BankAccount, command.BankName);

            //save new raised events of aggregate into eventstore
            _repository.Save(child, Guid.NewGuid());
        }

I have added a sample (without a GUI) to the sourc code. When you register a new child then a second command is send over the bus called "AddAssociatedPerson". Let's have a look at the method "btn_Save_Click" in "Sponsorship.Ui.Modules.Child.RegisterNewChildForm":

private void btn_Save_Click(object sender, EventArgs e)
        {
            //TODO: Format von Geburtsdatum überprüfen
            //TODO: Validierung (Mussfelder?)
            var childId = Guid.NewGuid();
            _bus.Send(new RegisterNewChild(childId, txt_Firstname.Text, txt_Lastname.Text, Convert.ToDateTime(txt_Birthday.Text)));

                        //Add an associated person to the child - no GUI for now - as a sample how to update an aggregate and the read side
            _bus.Send(new AddAssociatedPerson(childId, "Tommy", "Kakai", new DateTime(1980, 1, 1)));

            DialogResult = DialogResult.OK;
        }
The command AddAssociatedPerson should be filled by some textboxes in another from instead of the hard coded values. But this should be an easy exercise. Next open the Sponsorship.Domain.ChildCommandHandler. It's the same pattern (3 lines of code as stated above):
public void Handle(AddAssociatedPerson command)
        {
            //get child aggregate from the eventstore
            var child = _repository.GetById<Child>(command.ChildId);

            //execute domain logic
            child.AddAssociatedPerson(command.PersonFirstName, command.PersonLastName, command.PersonBirthday);

            //save new raised events of aggregate into eventstore
            _repository.Save(child, Guid.NewGuid());
        }
Let's have a look how to update RavenDB (the read side) in Sponsorship.ReadModel.ChildListView:
public void Handle(AssociatedPersonAdded domainEvent)
        {
            using (var session = _documentStore.OpenSession())
            {
                var dto = session.Load<ChildListDto>(Dto.GetDtoIdOf<ChildListDto>(domainEvent.ChildId));
                dto.AssociatedPersons.Add(domainEvent.PersonFirstName + " " + domainEvent.PersonLastName);
                
                session.SaveChanges();
            }
        }
Coordinator
Nov 12, 2011 at 7:34 AM

Hm, I'm having problems when trying to access an already saved child.

  • First I created the following Command and Event with the Contracts.tt T4 template

  ChangeChildFirstname? (childId, firstName)
  ChildFirstnameChanged! (childId, firstName)

  • Then I added this methods to the Child-Class:


private void Apply(ChildFirstnameChanged @event) {}

public void ChangeFirstname(string firstname)
{
    RaiseEvent(new ChildFirstnameChanged(Id, firstname));
}

  • Then I opened the ChildCommandHandler.cs and added the interface Handles<ChangeChildFirstname>.
  • I then added the following method to ChildCommandHandler.cs:

 

public void Handle(ChangeChildFirstname command)
{
    //get child aggregate from the eventstore
    var child = _repository.GetById<Child>(command.ChildId);
    
    //execute domain logic
    child.ChangeFirstname(command.Firstname);
    
    //save new raised events of aggregate into eventstore    
    _repository.Save(child, Guid.NewGuid());
}

 

  • And here is my Main-method:

[STAThread]
static void Main()
{
    InitializeInfrastructure();

    var childId = new Guid("339e838b-b6d5-42dd-bb0e-6c5e2fb55539");
    var bus = ObjectFactory.GetInstance<IBus>();
    bus.Send(new ChangeChildFirstname(childId, "MAX"));

    //Application.EnableVisualStyles();
    //Application.SetCompatibleTextRenderingDefault(false);
    //Application.Run(ObjectFactory.GetInstance<Form1>());
}


What I'm trying to accomplish is to load a child (after a fresh start) with all its past events applied, change it's first name and save the resulting new event in the event store.
I got the Guid out of the RavenDB Server UI (http://localhost:8080/raven/studio.html#collections/RavenCommits) and I have double checked that I really got the Id of a child object.

The problem lies in the line "var child = _repository.GetById<Child>(command.ChildId);" of the Handle method.
It seems, that the event store doesn't deserialize the events properly, because they are all in a state just after initialization.
Example: Image


@Jörg: I think this didn't happen in your AddAssociatedPerson example, because the EventStoreRepository keeps references to already opened streams (see CommonDomain.Persistence.EventStore.EventStoreRepository.cs:Line 85).
And as all the necessary events were still in memory I guess there didn't happen any deserialization.


Maybe there is something broken in our Event Store Wireup? Any ideas?

Coordinator
Nov 12, 2011 at 7:38 AM

Just for completeness the child event in RavenCommits:

{
  "StreamId": "339e838b-b6d5-42dd-bb0e-6c5e2fb55539",
  "CommitSequence": 1,
  "StartingStreamRevision": 1,
  "StreamRevision": 1,
  "CommitId": "cce26db5-a793-436c-8696-24ece99e7be5",
  "CommitStamp": "2011-11-12T07:42:30.3182552Z",
  "Headers": {
    "AggregateType": "Sponsorship.Domain.Children.Child"
  },
  "Payload": {
    "$type": "System.Collections.Generic.List`1[[EventStore.EventMessage, EventStore]], mscorlib",
    "$values": [
      {
        "Headers": {},
        "Body": {
          "$type": "Sponsorship.Domain.NewChildRegistered, Sponsorship.Domain",
          "ChildId": "339e838b-b6d5-42dd-bb0e-6c5e2fb55539",
          "Firstname": "Peter",
          "Lastname": "Alexander",
          "Birthday": "1901-01-01T00:00:00.0000000+01:00"
        }
      }
    ]
  },
  "Dispatched": true
}

Coordinator
Nov 12, 2011 at 10:24 AM

Found the problem:
Due to the readonly modifiers on all of our public event fields the deserializing logic couldn't set them. Removing them fixed the problem.

I added the source code of the Lokad.CodeDsl project in the External Libs folder and created a generator that creates our events and commands but without the ProtoBuf attributes and without the readonly modifiers. 

Coordinator
Nov 12, 2011 at 3:26 PM

Thx for your push to the source code repository.

Coordinator
Nov 13, 2011 at 12:21 PM

After long and intense phone calls with Adrian and Jörg I am PRO CQRS!
The main goal is a running version of the system on Nov 26th, but it should be fun too to implent and maintain the software. It will be stress and it will not be easy to build a system in one day, but who really wants to go the easy (and dirty) way?

My personal skills must not be the showstopper for a good system, I am willing to learn fast...

In other words: Yes, we can!!!

Coordinator
Nov 13, 2011 at 12:39 PM

*gone out celebrating* :-D

Coordinator
Nov 13, 2011 at 12:54 PM

stopped celebrating - gone learning CQRS...

Coordinator
Nov 13, 2011 at 12:57 PM

:)