How to explain EMF?

Have you ever tried to explain EMF to nonbelievers? I find it difficult to explain what EMF is and why it makes sense to use it. I had this a few times in my career:

• In the mid 80ies when I started with object oriented programming. For procedural programmers (modula2/pascal/C/Fortran) it sounded like a lot of buzzwords and seemed to add no real value...
  
• End of 80ies and in the beginning of the 90ies (before the GOF book was out), patterns seemed to be quite fuzzy and it was hard to explain what the value of describing "patterns" is.
  
• Around the same time I discovered scripting languages (starting with (g)awk->TCL->perl and ending with python, which I used for a decade). It was hard to motivate why those "slow" languages are in any way useful.
  
• Aspect oriented programming is still in the "hard to explain/motivate" phase.
  

The common pattern with those "hard to explain" new technologies is that they are incremental changes to existing stuff but once you understand and use them, they change the way you are thinking. If you adopt the "new technology" you feel a boost in productivity and you see better ways to solve old problems. The level of abstraction raises and you can focus more on the problem instead of dealing with low level implementation details (or reinventing the wheel). And then comes the time when you think all problems can be solved with the new "hammer"....

It is similar to converting to a new religion. For the believers it changes their life. It changes the way they see, experience and interact with the world. For the nonbelievers it looks like a stupid set of paradigms that make no sense. They learn how to block the arguments of missioners of the new religion and the more missionary the believers are the more skeptical the nonbelievers get.

So, how do you explain the benefit of EMF?
What is the best strategy to evangelize nonbelievers?
What is the best way to get the converts over the initial pain of change?

What are the typical questions and problems with EMF?
Here is a list of things I hear often:
Why to use EMF for my DSLs instead of some hand-written well tuned Java?
It generates lots of code and bloats my project.
EMF is so complicated, it takes a long time to learn -- in that time I have solved my problem twice without EMF.

What are scenarios where EMF increases productivity and where is it the wrong tool?

I have seen Peter Frieses talk at eclipsecon. I really liked it but I am not sure it helps nonbelievers to understand what EMF (and modeling in general) is. I think what is needed is a hands-on way with some real examples that show step by step how modeling and EMF can be applied to real problems.

I will post some of my experience with EMF in this blog in the next weeks and months.

The Eclipse Freeloader Award

The future of eclipse in in danger: The problem is that there is no real pressure for companies to contribute back to the community and it is easy to use the eclipse "for free" for the own products. There are some interesting blogs on this topic by Doug Schaefer on the future of eclipse and Bjorn on life at Eclipse in the new world.

According to wikipedia, this is called free loading: "choosing not to do work and letting others do it".

Eclipse is open source and companies can take advantage of the open source work. There is nothing legally that can prevent them from doing so. But the eclipse community should create peer pressure to prevent the freeloaders and parasites from getting away without punishment.

A few ideas to increase the pressure for freeloaders:

• Create an eclipse freeloader award
  
• Have an "eclipse supporter" logo
  
• Have a list of of freeloaders prominently on the eclipse page
  
• Create an eclipse pillroy

Is this too harsh? Well, it is less harsh than to take advantage of the work of others and letting eclipse die. It seems that positive stimulus is not enough. I want eclipse to succeed in the future. But if everybody is only thinking about his own agenda the entire system will die. Eclipse has to defend itself. The eclipse foundation is to polite to upset some companies. That has to change. It has to be an honor being part of eclipse, and is has to be crystal clear that eclipse is not for free!

Disclaimer (not sure if this helps, but it seems common to add a disclaimer to anything that could potentially upset an employer): This is my personal opinion and not the opinion of my employer.

Good versus evil diversity - why the foundation must hire developers

In German we have two contradiction proverbs: "Gleich und Gleich gesellt sich gern" ("birds of a feather flock together" *) and "Gegensätze ziehen sich an" ("opposites attract" *).

When applied to couples, I think both are true but for different aspects: for social status, education, religion etc, coming form similar background makes things easy. But when it comes to character, being of different kind is good because my partner might have something I don't have and together we are more "complete".

Diversity in open source is similar. There are areas where diversity is absolutely vital and other areas where diversity has a very negative impact.

Bjorn criticized the "lack of diversity" in the eclipse community because most projects are "mono-vendor". He is right, we need more diversity here!

But there is another kind of diversity that is bad: if there are many ways of doing the same thing. Excellent system are not designed by committee, but by a few people (often a single person) with strong ideas about architecture, design and the "right way of doing things" (Frederick Brooks calls this "Conceptual Integrity"). This leads to excellent systems. From my experience I learned that it does not matter much, which set of rules are applied, as long as the rules are consistent and everybody in the team believes in them and follows them.

Startup companies often get this kind of consistency "for free", because they grow form a small set of people and they hire only new people that fit into the "mind set" of the team. Open source projects, created by some enthusiasts, get it, because they attract people who believe in the system. The (original) eclipse platform is an example of such a consistent system.

I am afraid that the future of eclipse is in danger if this wrong type of diversity (I call it chaos) increases. (I wish the architecture council could help to defeat this kind of diversity, but I am afraid that the council is already way too diverse when it comes to "architecture")

It would help a lot if there would be a set of developers paid by the foundation coming form different backgrounds but with a common mind set when it comes to technology. If the foundation would hire people, I am sure we could find a great team of people that would be able make eclipse ready for the future.

Having a "vendor neutral" team of developers dedicated to the future of eclipse would give eclipse a boost. Else, I am afraid, eclipse is "auf dem absteigenden Ast" ("heading south" *)

New Book: Eclipse für C/C++-Programmierer

I just got a copy of Eclipse für C/C++-Programmierer by Sebastian Bauer, the first German CDT book. The first impression is very good. This book is ideal for C/C++ developers who are new to eclipse and new to CDT, but it also has a few gems for more experienced users. It covers a wide range of topics (see the table of contents), it is well written and takes a very good route to get started with CDT. Although it has a lot of screen shots, I don't have the impression there are too many, because they complement the written text in a very good way. It has a very good balance between practical tips and explaining the concepts and ideas behind CDT.

This book is the missing manual for CDT! It should be delivered with each copy of CDT. I wish it would be available in English because it deserves a wide audience.

eclipse 3.5M6: linked extension locations now require a .eclipseextenison file

UPDATE: the problems have been solved/explained (see below)

There is a hot debate between the community and the p2 team on supporting extension locations. A few weeks ago I described a simple procedure to use and maintain extension locations with p2.

In eclipse 3.5M6 links file do not work anymore, unless you add a .eclipseextension marker file next to the plugins and features directories of the extensions location (see bug 268673).

This marker file was required by the classical update manager in order to be able to add a directory as product extension from the UI. Until 3.5M6 link files just worked when pointing to a directory containing a plugins directory (or a eclipse directory containing a plugins directory). No .eclipseextension file was needed.

Ironically, the classical update manager in M6 cannot be used to add extension locations anymore. (To enable the classical update manager: Preferences...->General->Capabilities->Classical Update)

Is this a sign that the p2 team wants to discourage extension locations and link files? Or am I believing in a stupid conspiracy theory?

-------------------
Update! Both problems have been solved
1. bug 268673: p2 tries to be clever: when it detects a artifacts.xml in the extension location it needs a .eclipseextension marker file to believe that this is really an extension location. Normal extension locations are not affected by this bug.

2. bug 268674: This happens only if you never have started eclipse from the installation location (and always start with the -configuration option.

==> no conspiracy :-)

p2: how I install plugins in extension locations...

The following procedure explains how I install plugins into different extension locations and share those extension locations between different eclipse installations using the p2 update manager and link files. For a lengthly discussion see bug bug 224145

I keep a set of different extensions locations in C:\eclipse\extensions (each extension location is in a subdirectory of that directory). Suppose I want to install foo into the new extension location C:\eclipse\extensions\foo.

1. I start eclipse with:

   -configuration C:\eclipse\extensions\foo\eclipse\configuration

   
   
2. I install foo using the update manager.
   
   
3. Now I can use the extension by creating a foo.link file inside the links directory of an eclipse installation (you might have to create the links directory) containing:

   path=C:/eclipse/extensions/foobar

   
   

That's it :-)


Note: You have to the forward slashes on windows in the links file.

Note: This procedure works for eclipse 3.4 and 3.5.

Note: You don't have to create any directory specified by the -configuration parameter. Eclipse will do that for you.

Note: It is important to follow the pattern EXTENSION_NAME/eclipse/configuration for the -configuration parameter because p2 will put the plugins one directory above the configuration directory and link files require that the directory that contains the plugins is called eclipse.

Note: With this structure I can update plugins into the extension locations by running eclipse with the -configuration and then do the update.

Advanced use: If I want to install bar based on my foo extension, I create a link file to foo in the bar extension location links directory:

C:\eclipse\extensions\bar\eclipse\links\foo.link

and follow my standard procedure described above (the links directory can be populated before the first run of eclipse)...

Note for eclipse 3.4: If you are using eclipse 3.4 and you want to use the eclipse default update sites, you have to run eclipse without -configuration and export them (Help->Software Updates->Available Software (tab)->Manage Sites->Export), so you can import them into the -configuration eclipse.

Note for eclipse 3.5M6: You have to add an (empty) .eclipseextension file next to the plugins and features directory of the extension location.

When the runtime Application uses the wrong plugins...

Today I run into a problem that can cause a lot of frustration: a runtime eclipse launch that runs with he wrong plugins. The frustration comes form the fact that your application behaves strangely and it is hard to guess that this is caused by loading some worng plugins. Those plugins might even be closed or deleted from your workspace!

Here is what happens: PDE (eclipse 3.4) does some hidden magic when you run an eclipse application (lets call the launch configuration foo): it creates a directory in .metadata/.plugins/org.eclipse.pde.core/foo. This directory is used with the -configuration flag when running the application to store the runtime configuration of your application. But, there can be cases when this directory is not updated correctly, like when you switch between "launch with all enabled and workspace plug-ins" and "plug-ins selected below". It can happen that the runtime application still uses all plugins although you have selected to run with some specific plugins. I have not tested the exact conditions when this happens, but it happened several times to me in the last months.

Unfortunately the "Clear workspace" flag does not help, because it clears the workspace but not the configuration area! It took me several hours to figure that out....

As Chris points out in the first comment, here is the solution: On the "Configuration" tab of the launch configuration check the option "Clear configuration area before launching".

Chris also recommends to use -clean -consoleLog -debug -console as arguments for plugin development launch configurations. In this case, the -clean option takes care of cleaning the configuration area.

Tip: you can set -clean -consoleLog -debug -console as default in Preferences->Plug-In Development->Target Platform->Launching Arguments->Program Arguments

I hope this tip saves someone some wasted time...

.

Finally, I found a good book on JavaScript!

For some years I have been looking for a book on JavaScript that would explain the language to someone who already knows programming. But all I have seen until recently are books for beginners or books that talk a lot about web programming, web browsers, DOM, XML, HTML etc, or books that explain all the different versions of JavaScript implemented in different browsers (many of these books have 1000 pages or more).

The book I read is a 180 page book called JavaScript: The Good Parts by Douglas Crockford. The idea of the book is: JavaScript is a messy language but here is a subset and some best practices how to use JavaScript in a manageable way. And that is done on 100 pages (the rest are interesting appendices). I read it last week on a flight and it was a kind of eye opener for me.

I finally understood that the object system of JavaScript is not class based like in C++/Java/Python/Ruby but it is prototype based like in Self/Perl/Lua. This prototype based approach is not supported by a special syntax. There are many ways of creating and dealing with objects. But be book makes some suggestions and explains some best practices for creating objects.

The other eye opener was how closures can be used to build objects with private information.

An interesting design decision of JavaScript is that there is essentially one data structure, called Object, with string keys and any object as value. It can be used as hash map, array or object. Even functions are objects and you can associate information with them or add methods to one or all functions.

If an object is used as array, the index is converted to a string key -- I find this very strange -- but it seems to work. The array.lenght attribute is defined as the "largest integer property name plus one", which means it looks for string keys that look like positive integers and takes biggest one, provided it is less than 4294967295.

If you want to learn JavaScript as a language I would highly suggest to read JavaScript: The Good Parts. After you finished the book, you might not be able to read any bad JavaScript programm, but you might be able to write your own "clean" scripts. I really like that approach.

If you are interested in E4, knowing JavaScript might be useful, because E4 will be scriptable with JavaScript.

Eclipse summit europe -- no more hotels

OK, I admit, it is a bit late to look for a hotel for Eclipse Summit Europe 2008. Today I tried about 30 hotels. It seems all hotels in Ludwigsburg are booked out. Well, I thought no big deal: I would follow the S-Train line and book a hotel close to any S-train station. Well, all hotels are booked out along the S-train lines, including the hotel close to the Stuttgart main station!? I finally found a hotel not far from Ludwigsburg, next to the S-Train station in Asperg, but it turned out that the address was listed wrongly in google maps and the hotel is not even close to any train station....

It's not that bad -- with public traffic it is 1h30 to 1h45 from my home in Heidelberg to the "Forum am Schlosspark". The only bad thing is that my latest train is around 9:30PM, and the good discussions are usually late in the bar....

That means waking up early (it's really early for me, since I "virtually" live in the EST time zone -- I go to bed at 6AM and wake up at 1:30PM)...

If you found a hotel in <30 min distance to summit, let me know.

ConcurrentModificationException: Why do Java collections not have robust iterators?

Ever got a ConcurrentModificationException? It just hit me. How does this happen? It happens when one method iterates over a collection while another method (that bis recursively called from the for loop) modifies the collection.

Note: ConcurrentModificationException has nothing to do with threading! (Well, this is a bit too strong statement, as Rafael points point out in a comment: it might occur due to a threading race condition (but in such cases you probably have other problems as well)). A ConcurrentModificationException may have nothing to do with threading!. Here I am talking about about the non threading related case.

Here's a simple example to show the problem. We have a class that can register new listeners and when fireChange is called it calls changed on the listeners. So the test listener here removes itself on the change call and boom, we get the ConcurrentModificationException:

public class IteratorTest {
 final Collection fListeners;
 public IteratorTest(Collection listeners) {
  fListeners = listeners;
 }
 static class Listener {
  public void changed(IteratorTest subject) {
   subject.removeListener(this);
  }
 }
 public void addListener(Listener listener) {
  fListeners.add(listener);
 }
 public void removeListener(Listener listener) {
  fListeners.remove(listener);
 }
 public void fireChange() {
  for (Listener listener : fListeners) {
   listener.changed(this);
  }
 }
 static void test(Collection coll) {
  IteratorTest t = new IteratorTest(coll);
  t.addListener(new Listener());
  t.fireChange();
 }
 public static void main(String[] args) {
  test(new ArrayList());
 }
}

The problem can happen if you call out to "other code" (code someone else has written) and "other code" can change the collection while you are iterating. One solution is to iterate over a copy of the collection:

 public void fireChange() {
  Listener[] listeners=(Listener[]) fListeners.toArray();
  for (Listener listener : listeners) {
   listener.changed(this);
  }
 }

That helps. But there is a lot of code out there that iterates over a collection and calls "other code" and there is always a chance that the "other code" calls back to modify your collection and you get a ConcurrentModificationException....

The good news is: Unlike threading race conditions it happens deterministically. The bad news is: if you are the client it is often not easy to find a way out.

15 years ago, ET++ (the cool framework created by Erich Gamma and Andree Weinand in the 80ies) suffered form missing robust iterators. "Robust iterators" means robust to changes of the underlying collection. At that time, making a copy of a collection seemed an unacceptable overhead. So, Thomas Kofler added robust iterators to ET++ (the PDF has the pages on reverse order -- here is a readable version). The implementations are efficient and robust.

Robust iterators are so fundamental, I am really surprised that Java does not have them....

.