A Blog about Business, Enterprise Architecture and Software Design by the Founders of ZenModeler

Know The Trade-offs Of Your Design Decisions – QCon London 2012

Trade-off is the keyword I remember the most of that first QCon London 2012 day. It was a key point of Dan North’s presentation yesterday: each decision imply a trade-off, wether you know it or not.

QCon Logo

Trade-off means that there is no black and white choice, you always have a trade-off, the point is to make an informed decision not one influenced by emotion or fashion.

Dan talks about 4 topics and the usual decisions that goes along:

  • Team composition: co-located or distributed ? feature or layer team ? experienced or inexperienced ? small or big ?
  • Development style: automated or manual build ? test-first ? TDD ?
  • Architecture: monolith or components ? resources or messages ? synchronous or asynchronous ? single event loop or multiple threads ?
  • Deployment: automated or manual deployment ? vertical or horizontal scaling ? hosted or in-house ? bespoke or commodity ?

Common Sense is better than Common Practice

Dan North.

This is a quote when he talks about times when tests are the worst thing to do given the trade-offs: longer time-to-market, disposable application, low return on investiment, lower agility for change, etc.

Design is all about the decisions you made

All your architecture is about the informed decision you made. Your decision can have a system wide impact in the quality attributes of the system or a localized one, that define respectively macro (cross-system) and micro (system- internal) architecture as Stephan Tilkov stated it in his presentation. Design is all about the macro and micro level decisions you made.

Stephan also talks about the objectives and rules that goes with each architecture. Here are some topics that can have rules associated with, each one categorized as macro or micro:

Cross-System System-Internal
Responsibilities Programming languages
UI Integration Development tools
Communication protocols Frameworks
Data formats Process / Workflow controls
Redundant data Persistence
BI Interface Design Patterns
... ...

To take account of time, each set of rules is versionned. With the assumption that two systems in the same Rules version can interoperate seamlessly.

In the beginning of a project the objectives are more the following: ease of development, homogeneity, simplicity, etc. and as time goes by (and as the artefacts increase) the preponderant objectives becomes: modularity, decoupling and autonomy. Nice point : Stephan talks about the domain architecture, that is often the most stable one.

Finally, Stephan talks about integration: especially UI and data. I left apart the UI one to focus on the data one with that pieces of wisdom shared by Stephan, Greg Young and Dan:

Each time you mutualize something between two components (code, data) you introduce a new coupling

 

The inverse of « DRY » (Don’t Repeat Yourself) is « Decoupled » !

That’s particularly true for data replication, often considered bad but replication permits more autonomy and avoid synchronous calls that directly impact availability and reliability. The solution is asynchronous: use events for notifying interested components of a change.

The second keyword I notice several times is « entanglement » (actually there are several keywords but I grouped them: cyclic dependencies, braid, intricacies). Reducing dependencies is a point that I promote for a long time. Dependency is often added as a micro level decision but every time one is added it augments the overall system complexity. I often use Structure 101 to manage the dependencies in the Java system I work with and I’m very satisfied with it.

To conclude that first day report I quote Rich Hickey, Clojure creator:

programmers know the benefits of everything but the trade-offs of nothing

Everything Fails All The Time : Cure Or Prevent Errors In Your Design?

In computing « everything fails all the time », like Werner Vogels, Amazon’s technical director, used to say – or, to put it more bluntly: « shit happens ». Instead of preventing at any cost it can sometimes be more worthwhile to cure, that is to know what to do in case of error. And not only from a technical point of view but first and foremost from a business point of view.

This will allow me to point out these « errors » and how instead of avoiding them it is sometimes better to think on a global scale about the role these errors play in the system.

In order to illustrate this view I’d like to describe the inner workings of the Starbucks Coffee Shops who have a reputation for not the quality of their coffee but for the sheer number of orders they are able to process every hour (otherwise known as the throughput). This is the opportunity to illustrate an important number of principles linked to distributed computing with an example from every-day life.

How does an order at Starbucks get processed?

Here is a user story and a typical scenario:

  • User Story:
    • As a client
    • I want to place an order
    • So as to enjoy my coffee as quickly as possible* Scenario:

The asynchronism introduced when the cashier transmits the order to the barrista even though the financial transaction isn’t over is what I find most interesting. But in this case, what happens if an error occurs during the process?

  • In the case of non-payment (not enough cash, credit card blocked, etc.): the coffee is simply thrown away or given to another client who has luckily made the same order.
  • In the case of a messed-up order (the cashier misunderstood my order, the barrista misunderstood, etc.) : the order is simply redone.
  • Etc.

The possible mechanisms in order to process an error

  • « Keep moving on »: when the consequence of the error is negligible (like a coffee for example)
  • « Try again » : useful especially when one of the links in the chain is momentarily indisposed. Example from every-day life: the cashier repeats the name of the order several times until he receives an answer from the barrista.
  • « Compensate »: what does my bank do when it has wrongfully charged me for a service by debit? It credits my account with the corresponding amount, so as to compensate the mistake.

Benefits

The asynchronism introduced after the order allows the increase of occupied time of each participant. The barrista does not need to wait for the end of the payment to start making the coffee. I’ve even sometimes noticed during rush hour at Starbucks how an intermediary actor will take the order in advance, playing the role of dispatcher between the two lines of treatment: payment and creation.

The consequence of an error in one or the other of the treatment chains is negligible in regards to the huge gains obtained in the debit (throughput) of the system. The important point to highlight is that instead of focusing on avoiding at all costs an « error » in the process, the errors are considered as part of the system with consequential actions which are provided for in case they arise.

The functioning of Starbucks is a good way to illustrate how instead of putting into place mechanics which avoid any errors by the introduction of a centralizing element (no coffee gets done if the payment isn’t processed first), it is sometimes better to accept errors in order to increase the occupational rate of each actor in a system. And so every potential error is regarded through its consequence.

This mode of functioning also puts into gear the principal of correlation identifier: that is how I correlate two treatments which are executed independently in order to finalize processing in its entirety – like how the client receives his coffee after paying for it in our example. It is then necessary to correlate the client, the payment and his order. This is done in several different ways depending on the Starbucks establishment and the country, the correlation identifier can thus be:

  • The name of the ordered coffee: especially in Europe
  • The client’s name: I have experienced this mostly in the United-States, it should be noted that this reenforces the homely aspect of Starbucks because I much prefer to hear my first name called out instead of an impersonal “who ordered the double expresso Machiatto?”

Conclusion

I like this example from Starbucks because the transposition into the real world of a mechanism and the intuition that goes with it are an excellent way to conceive or comprehend something which can otherwise remain obscure. This also allows us to illustrate the mechanisms used by distributed and asynchronous systems:

Ultimately it’s mostly the opportunity to demonstrate how a globally integrated “common” error management of the entire treatment chain permits an optimization which leads to considerable gains in debit in processing.

This article was inspired by the formerly published article by Gregor Hohpe « Starbucks does not use two phase commit ». I also recommend the following article by Gregor: « Let’s have a conversation » in which he compares the exchanges between two systems to a conversation between two people.

Class Or Objects Whats The Deal

Recently there have been a lot of articles published and frameworks built which debate about whether or not the notion of Class should be added to Javascript? In this article, I give my humble opinion on that debate and where my focus is : on the type system, not on syntactic sugar about the class keyword before my object constructor.

For those who instantly start to snicker at the mere mention of Javascript they should know that it is THE language of the moment and that it has very solid foundations from a theoretical point of view… well, ok, less so from a practical point of view (see « Javascript the good parts »), but node.js has been in first place on github in the end of the year.

I am surprised by this desire to add the class concept to Javascript because it seems to me to be a perversion, in the etymological sense of the word: « action to divert something from its true nature ». Indeed, the notion of class is absent from the principles of conception which make up the foundation of Javascript language, and they are:

As a reminder, Javascript is inspired by Self (I also recommend this excellent article « Self, the power of simplicity »), the first language to actually put into practice this principle of Prototype. I have been passionate about the object paradigm for several years now and I would like to add my point of view to the debate at hand, my blood turning cold from reading certain articles. Wanting to add the concept of Class to Javascript is in my opinion a non-issue and I will demonstrate this in the following sections.

Class, type and interface

A type defines an interface and a behavior, while a class defines a particular implementation of this interface and behavior. The class-concept is hence an implementation of the type concept. A type is defined by an interface and a behavior, that is its attributes, but most of all – and most importantly in my opinion – its methods. In Javascript, these notions of attributes and methods are stretched to encompass the notion of properties (or slots) without any sense of visibility. I like the visual aspect of the term “slot” because it illustrates the dynamic aspect of Javascript objects. And so, the class can be seen as the blueprint of the soon-to-be created object.

To introduce the class-concept in Javascript would be to deny its highly dynamic nature, in other words denying it the possibility to add during execution functions and attributes to any object. Nevertheless, class is interesting because it allows to join in the same spot the constructor, the attributes and the methods. What bothers me is how focus is put on an abstract concept (class) and how it is considered to be static although what’s really important is the object, which is extremely dynamic and hence potentially weakly linked to its class. Basically if you want static instead of class see Dart by Google.

How should a new object be built? the constructor

The most important element in object-programming is actually the object concept, and especially everything in relation to its construction. This is why the notion of constructor is so important to class-based language (Java, C#, etc.). In Javascript, the keyword “new” followed by the name of a function insures the mechanism of constructing a new object. This way the notion of class doesn’t appear anywhere in the language. Furthermore, I find it rather remarkable and amusing how in Java the only moment when the class name appears is behind the keyword “new”.

The concept of heritage allows an object to dispose of the slots of another object. The inheritance-concept being supported by the “prototype”-object  associated to the newly-created object (this works sort of like in a chained list) where the executing motor searches for unknown slots by reviewing a list of “ancestors” from the bottom.

Contract, classification and typing

How should an object be “classified” and how to be assured that an object belongs to a given type and so answers to an interface? Behind all this I believe sits the question of the behavior – “how is this object going to react faced with this message?” – in application of the substitution principle developed by Barbara Liskov.

Strong typing allows then to validate the interface, but only in its global signature and not in the behavior. Programming by contract concept provides this kind of mechanism of validating behavior before the execution. This isn’t available in Javascript so I would rather turn to unit test frameworks which I use in a Behavior-Driven Development sauce and that I sometimes summarize by “programming by contracts within tests.” In CoffeeScript the Contracts.coffee extension has very recently made this mechanism available directly.

My ideal language? One based on prototypes, strongly typed and dynamic

In an ideal world I would like a language based on prototypes, strongly typed and dynamic. What is important to me is:

  • How do I build an object and how do I define its attributes and the functions linked to its changing states?
  • What is the behavior of an object at an instant t in time and based on the context?  Well yes, I am interested in the Data/Context/Interaction pattern derived from the works of Trygve Reenskaug, the inventor of the MVC, and James Coplien. Javascript’s dynamic nature is perfect to put into action the mechanism of trait or mixin, allowing the composition of different behaviors (hence functions) at the heart of an object depending on the context.
  • What is the contract of this object and how can I assure the correct use of it? possibly before code execution

Python is one of the closest language from that ideal, but I find the new kids on the block (Dart, Contracts.Coffee inspired by Racket, etc.) to be smart enough to gently allowing introducing strong typing when one has the most need of that feature.

The mechanisms that I put into place in my programs in order to ensure their robustness rely heavily on contracts. In Javascript, I don’t have such mechanisms, so I do differently : essentially I check through testing the asserts and contracts that I expect to be true within my code. But I am really eager to try that new languages.

Conclusion

To summarize, I find Javascript very healthy and very useful with palliatives that are in the end easily put into place. With my experience, I have developed my own style of design and programming that fits me very well by dipping into concepts and not into implementations and I always manage to find what I’m missing. So, in a nutshell, I don’t need classes but a richer type system and for me the future is bright given the pretenders. What about you? What do you need most from your everyday programming language?

Update the Feb 10 :

This discussion about « untyped » on Stack Overflow give a lot of clues about that subject.

Identity Data And State The Fundamentals Of Object And Functional Design And How To Manage Them

State management is a subject which is currently at the forefront of the debate in the software world. The functional paradigm as well as the reflection on the basics of the object paradigm make it so non-mastered changes in state are seen as the root of all troubles. In this article I will return to the notions of state and identity because I consider them to be crucial. What is a an object state ? What is an identity ? These concepts do apply whatever the used paradigm is, whether it’s object-oriented or functional-oriented.

Identity State Behavior

When I started my professional career as an intern, I had a discussion with my project manager over a UML model (she had a PhD in computing and was specialized in object-oriented stuff) and her first question was “But where is the dynamic part? Why isn’t there a state-transition diagram?”. It was a complete revelation for me and initiated a deep change in my approach of software system modeling: the behavior, what the system does, is just as important as the structure, what the system is. This balance between the static part and the dynamic part must always be respected in order to obtain a design in its rightful name, proof of a good maintainability. Some people even speak of the tao of programming, a balance between the Yin (the state) and the Yang (the behavior).

Data

Before tackling the notion of state, I’d like to come back on the notion of data which, composed through the object attributes, constitutes its state.

Data is a couple formed by a concept and a measure.

The concept which the data is referring to must have a precise definition making it possible to rally to its semantic.

The type of measure can be extremely varied, but can often be categorized as follows:

A data can be fixed (immutable) or not (mutable). The measure of the data changes all the time, so the value of the object attribute can be changed. This frequency of change can be elevated or not. It can be very visible or not. These different attributes allows to characterize each data and can be summarized in the following diagram:

Example

Different kind of data characteristics

This reminder of the notion of data is enough to start tackling the notion of state.

State

A state is the sum of data values which characterize an object at a precise moment in time.

In the widely accepted object-oriented paradigm the state of an object defines its structure. This state is important for the object because it will influence its behavior during a future interaction after a stimulus.

The « state » characteristics are as follows:

  • A state is the sum of the data values characterizing an object at a precise moment in time. An object is characterized by a group of attributes which are composed of data (concept and measure).
  • A state is sometimes summarized by a past participle with a static and stable signification.
  • A state lasts as long as no external stimuli interacts with the object holding the state.
  • The state influences the behavior of the object.
  • A state, in its definition, implies that there are transformations of the object between different states which intervene during the transitions of state.
  • A state is associated with a particular object which has an identity, in order to follow the different transitions of state.
  • A state is always signified by an attribute whose data is discrete and variable. By discrete I mean a continuous attribute but a discrete continuity. For example: the state “majority” of a person is discrete (adult, minor) but is obtained through a continuous attribute – the age of the person.

Example

The following diagram illustrates the states that can undergo a person:

The different states (not exhaustive) that a person can be at a given time

Every transition is triggered by an event which leads to the transformation of the considered object. Continuous data like age can be “subdued” by a threshold condition like the passage from the adult age and so can be transformed into a state.

Identity

Identity is what makes it possible to distinguish between oneself and the other. It’s a basic biological principle. For that matter, Alan Kay established his entire vision of the object paradigm on biology:

Alan Curtis Kay Alan Curtis Kay source

I thought of objects being like biological cells and/or individual computers on a network, only able to communicate with messages (so messaging came at the very beginning – it took a while to see how to do messaging in a programming language efficiently enough to be useful)

The concept of identity for a person seems very natural to us. This identity starts at birth and ends in death, but can even go beyond that. What’s interesting is the transformations which undergo this person continuously: his age, his financial situation, even his name can change while at the same time maintaining that same identity. The object attributes change but the identity stays the same.

From the programming point of view, different objects can co-exist, with different attributes, within the same identity. It’s typically the case for a client, who can co-exist in very different contexts: in a marketing context to get to know his consumer habits, an accounting context to handle his payments, a logistical context to handle mailing, etc. Nevertheless, his identity will need to be precisely defined in order to be able to link different objects between them, representing each different aspect of the same identity. Does it matter to the software user to consider that this identity evolving in time is always the same one? Or is it acceptable to consider that they are different? It’s the main question that the creator must answer, because any software runs on a model, which is then only a representation of reality.

This leads us to the concept of entity which is defined as the association of an identity and a state.

An entity is the association of an identity and a state The object paradigm fuses these two concepts in the object, the functional paradigm isolates them, their fundamental differences are reached. The notions of identity and state, which constitute the entity, are nevertheless eminently temporal as shown in the following diagram:

Identity and State evolution across the time for a particular object

The identity is the invariant of the object in time and it unites the sum of its attributes, that is its state. In the functional paradigm, this notion of references from the identity to the attributes is the foundation of the persistent data structures. The functional paradigm brings this temporal dimension which, I think, is lacking in the object-oriented paradigm and brings a great ease to concurrency management.

Put into practice

Manage the states and their transition

Every object with a strong identity must provide the operations allowing the manipulation of its state in a controlled manner:

  • All the state transitions must manifest themselves through an operation on the object: for example, the operation “to divorce” on the object of a person.
  • Operations must make it possible to interrogate the state of the object (the famous “getters”).

Manage the identity

Every entity must provide an identification which is unique within the whole system even if its attributes are identical to another entity.

The identity needs above all a real reflection during the course of conception on the notions of difference and equality between objects. This notion of identity is part of the model and so every identity must define precisely its unifier and its equalizer, even in free text. Always be vigilant during exchanges with experts in this field on the equalities linked to object attributes that can hide a lack of reflection on the identities.

In certain cases, the identity will be composed of the combination of several attributes. For example, an episode of a TV series will be identified by its title, its season and the episode number (ex: “Numb3rs-Season6-Episode13”).

In other cases, a unique and unchangeable identifier will be generated and attributed to the object. This is the usual client number on your electricity bill. The generation of these identifiers is an entirely different subject from the technicality that it implies (concurrency, distribution). Finally, the generation of identifiers implies a reflection on the borders of the system in order to avoid collisions and in the course of any exchanges with another system, like when a person is identified by his social security number.

The attribution of the identifier often reveals the focus at the heart of the information system, and so telephone numbers as identifiers and not the person make any common actions difficult in relation to a client who posses several phone numbers.

Conclusion

In conclusion, the fundamental questions in an object design are:

  • What is it that makes two objects be considered identical or different?
  • What are the attributes which compose the object? What are the characteristics of these attributes? Most notably which are the ones that influence the behavior and how do they evolve in time?

These questions in connection with the identity and the state of the object are crucial for the realization of the domain model, which is at the base of any enterprise architecture project.