Cool DSL with Groovy

Domain model design has never been confused with ‘ease’. From the dawn of its conception, generating executable Unified Modeling Language (UML) diagrams meant sweat and frustration. Generating stubs & skeletons, alone, led one into quagmires of Java Enterprise Edition. Yet it is inherently possible — and actually has been for ages — to design a programming language dedicated to solve specific domain problems; effortlessly and quickly.

DSL is not a foreign thing. Developers are steeped in DSL, even if unwittingly. Frameworks are DSL. A macro is also DSL.
When you write a function? That too is steeped in DSL. But functions are often a dirty business; un-standarized; quirky, whereby even domain experts face a daunting task when unraveling and then being put to task to update such rapscallion boilerplates (that is if they do not themselves compound the problems).
A programmer worth his reputation would find it necessary to isolate and clean up all non specific domain terms within the DSL. This literally means rooting through and setting aside all secondary-in-importance setup code, allowing the domain expert to establish or reestablish the primary foundation code.
This is most times done for large money for an end user; the helpless customer.

Given an appropriate DSL that fits their needs, customers could write all of the
code that they need themselves, without having to be programmers.

Now let us imagine, or better, take for granted, that a good DSL experience can exists.
That given an appropriate and standardized DSL environment, even the customer,
with a few simple tools and grasp-of-concept, can self-write code, gracefully, and
specific to their needs without the cost and hassle of having to become, or running to,
programmers. And that is where Groovy comes in.

What’s Groovy? It’s Java as it should be. Easy and intuitive, it offers new features unknown to its parent yet (I’m still waiting for release 7), and come up with those benefits that form the basis of the DSLs that we will develop.

A fundamental feature is the MOP, the ability of changing runtime the properties and the behaviour of objects. It allows us to respond to method calls that do not exist in the class, in other word to “pretend” that these methods exist.
Another essence to consider is the Closure. It’s the real power of Groovy. The extreme flexibility of this kind of object/method allows us to change its behaviour replacing its delegate class, on the fly.

Closure delegate. Groovy has three variables inside each closure for defining different classes in its scope: this, owner, and delegate.
The this variable refers to the enclosing class. The owner variable is the enclosing object of the closure. The delegate variable is the same as the owner, unless that delegate is substituted.

class MyClass {
def closure = {
println "this class:"
println "delegate class:"
def nested = {
println "owner nested class:"
def tryme = new MyClass().closure
tryme.delegate = this
this class:MyClass
delegate class:Script1
owner nested class:MyClass$_closure1

When a closure encounters a method call that it cannot handle itself, it automatically relays the invocation to its owner object. If this fails, it relays the invocation to its delegate. one of the reasons builders work the way they do is because they are able to assign the delegate of a closure to themselves.


MDA on fire off the Shoulder of Orion

I must admit, frustration has increased over the years. I mean, interacting with the computer in terms of boolean, long, void. I’d rather sit on a sofa and describe the program by voice, or better, get into a 3D virtual reality and cook a software like a lunch meal in the kitchen. Playing with spheres, arrows, to design all program doodles.
I can’t picture it as a possible scenario in the near future, and like for the science fiction movies, we will need to wait a much longer time compared to what envisaged by movie directors or writers to see a minor part of the technologic developments imagined so far actually implemented.
It’s just for the secret ambition to get free from the textual codes that I’m debating about how to easily abstract the definition of information systems. In some ways, I’ve managed to do something related to this, in a restricted domain.

When Model Driven Architecture turn out right
Once during an interview, I was asked why I didn’t apply MDA over all software projects I leaded. The question should be hooked as a point of discussion on MDA misconceptions, and generally, on modelling and UML. Unfortunately these interpretations are hindered by a phenomenon that a famous observer of human events (Mark Twain) revealed, that I repropose it again in IT terms:

People commonly use UML like a drunk uses a lamp post; For support rather than illumination.

The initial costs of an MDA are pretty high, the return of the investments starts at the beginning of the automatic code generation. The models are built on the meta-model basis which defines the semantics of the system. Afterwards, the models are turned into code or other resources ready to be installed into the real system.
The initial developing efforts are focused on the meta-model and on transforming tools. The investments are rewarded by the automatic transformations that replace the repetitive coding work of programmers; and deeper is the amortization, the more the investment is profitable.
The ROI increases as much as the model instances are built, as well as the simplicity (in other words, easier to implement) of the metamodel and the trasformation tools. Where shall I apply this approach for best results? In a real-time video application? In a powerful compression API? I don’t think so. I guess high values of this coefficient would be found into SOA systems.
MDA is suitable for service domain, like a banking middleware, where you may amortize the modelling system through hundreds of services, with many data structures and flow descriptors, but conformed to few abstract structures, the metamodel actually.

Models are not code
Do you have to adopt UML for design metamodels? Definitely, not. You may define simple data classes in any textual format. The matter might raise when the metamodel grows in complexity, size and when many aspects of the domain are schematized into. Hence, you have to face UML: either you reinvent it, or you use it.
Perhaps tracing circles and arrows would be more embarassing compared to typing on a keyboard, but it could be necessary when structures become hinged and interrelated. Try to join a class diagram with a sequence diagram, then enclose all in a composite for interaction with external parts, and do it without formal and conventional visual patterns and… let me know!
UML is a complete and exhaustive, it’s so generalist it would be applied to describe any software model, but for describing not for coding.
Many people confuse UML as a programming language. Wrong. It’s a tool for representing a system, a structure, a flow. As mathematics aims to formulate conjectures among countable entities, UML offers a way to define abstract entities. Such high-level language is a mere conceptual schema, it defines components and services, no programs ready to run.

MDA layers
So what do you do with a picture with bubbles and arrows? Is it enough design some diagrams, push a button and voilà: getting a working system that fit your requirements? Nobody believes it, neither myself. The object model is unaware of the underliyng system and of its implicit matters. As we know, the model declares the structure and the behaviour differences between one service and another, for all the remaining aspects the system applies common platform specific beaviors. For instance, whether you want to enclose the service into a transaction you may setup a ‘transaction’ stereotype in your own profile metamodel, which will be transformed into a java annotation or xml attribute and then properly interpreted by the server framework. Once the MDA is ready, one designs the models and trasforms them into hardcore resources, the mythic code bullets, finally deployed into the server.
It would appear as too simple, and in order to keep off prejudicial comments I’ll tell you that the real applications are much more complex than this simple vision.
You can’t update code bullets by hand because the changes will be lost next time you generate them, automatically. Many exceptions are to be considered, allowing for example, to update generated code. I’ve used merging API like JMerge, and I found it useful to enrich the code without discontinuity from the model and the generated code.

7 steps to MDA revolution

scrumI didn’t believe that such a successful project was such a rare event in the IT industry, that’s why I’ve never caught another chance for applying the learned lessons again. I thought that the experience accrued on Model Driven Architecture will be reusable in other circumstances, though I’ve never seen concepts as executable UML or MDA either applied or mentioned in the following commitments I’ve pursued into.
The idea of this project wasn’t conceived by external consultants thirsting for selling their cool technology; instead, it was born and grew up just inside the development team. The architecture’s transition had been gradual, and little by little, as new automation scenarios penetrated our excited minds, we moved as many as possible development processes under MDA framework.
Despite my early impressions while considering to undertake the project, the upper management embraced it and laid down investments counting on the benefits that this new approach would provide to the development.
What is difficult to change is the modus operandi of a 300 employee company that offers banking services and applications, which is engaged in one of the most conservative field in technology and development methodologies by default. It was about a significant jump in the services development and as the PM remarked: “We are developing as dinosaurs, don’t you know what the hell happened to them?”, the way to MDA was traced.

The issues we faced with the introduction of modelling notions would be defined as practical contingencies rather than theoretic or philosophical reasons, foremost the mess in the business layer. It raised reliance and maintenance weaknesses with time, even security holes that sounded so bad in such a company with a plenty of  banks as customers.
The hundreds of cases developed by dozens of engineers turning over throughout the months in the java development area had reached the critical mass, enough to trigger an explosion/implosion of the whole system. On the other hand, the applications can stand up only by high costs of maintenance and lazy deliveries, due to the difficulties on integrating incoming services with the underlying system.
The application layer managed the data flows between clients at the top and feeds and legacy information systems at the bottom. On their way, they affected several mixes by business process rules hardcoded in obscure java classes. Unfortunately, most of those shaking details were lost, because of the policy related to the development, which didn’t claim about missing documentation, and then it was so damned annoying to go back and take over old artefacts for maintenance or updating rules. Only skilful programmers might extricate the balled up code. The critical mass had to drop down and be brought to lower temperatures quickly. New developments and dozens of  incoming features were planned, so a deep refactoring was a must; it can wait no more.

How was the domain layer implementation that popped up the highlighted problems?
The developer’s effort was mainly focused on the creation of java classes implementing a Command and defining the service to the framework through an xml descriptor. The input and output of such a command was a raw DOM argument, which was parsed to extract the input data needed by the business transaction, the most part of coding was regarded for parsing and filling the response’s service that was a raw xml document too. I think it isn’t agreeable to put most of the developing efforts merely on managing input/output data and mapping, but this was the daily job.
Apply the MDA take time, it was an one year evolution, and it would be summarized with:

  1. XSD barriers. It was necessary to set some boundaries for developers, in order to get a minimum of control over the data flows. Each service had its own formal validation on input/output data, though no restrictions were settled on how implementing the services. Never ever elements or attributes not defined in advance by commitments.
  2. Pojo. Replace the raw document with simple pojo as an argument in the call-back methods; this operation aggregates the formal validation with an easy approach on data manipulation. The binding xml-java isn’t hurdle, it is automatic and many available libraries can accomplish this step.
  3. First hints with EMF. Xsd files are models for xml data, EMF is a MOF java implementation, a general abstraction for writing all sorts of models, I don’t linger over it now, but it represented a jump to the service modelling. EMF is an open source library enclosed in the Eclipse platform easy to use and customizable, it aims to separate the abstract model from the ground.
  4. Choice of technology. The play with EMF opened new horizons on modelling facilities. Hooked by this methodology to design SOA applications I realized EMF is not enough, the UML (which core principles are inherited from MOF) can fit much better with my purpose to design the object model, define the process flow and the user experience. UML offers diagrams that you can join together, static and dynamic model may describe most of application structure and behaviour.
  5. Executable UML. What do you do with this bunch of diagrams if you can’t transform them in real artefacts and plug-in them in your SOA framework? Not so much, keeping UML diagrams without related transformations and executions is merely fine for documentation, not much more than this. At that time the company joined the Rational beta-program and I started to develop Eclipse compliant plugins which leveraged the power of UML2 eclipse implementation.
  6. How to define data mapping? One of the main obstacles encountered was the data mapping between two different structures. It happens when you need to connect two or more components inside a service call, and each of those have different data structures. In this case UML doesn’t provide any help and you have to customize the model with special stereotypes and profiles.
  7. Sequence and state diagrams. Class diagram were used to generate java classes, xsd files, copy cobol. Sequence diagrams on the other hand describe the flow of processes and their business rule, even conditional instructions which may be transformed to bpel or custom service descriptors. State diagram shows its benefits modelling the user experience and the steps to complete an operation, it easily tracks the state of sessions and will be transformed into the MVC system, as well as in whatever rich client forms.

Have your say.