Thursday, May 29, 2014

How to make eclipse attractive to new communities?

Recently, I had some discussions with different people on how to make eclipse attractive for non-java communities, in particular for web and mobile developers. This is also based on my experience doing web development. Here are some of my thoughts:

The Java mind-set in eclipse

The eclipse IDE is for many people (like me) a super powerful Java IDE, and in fact, it killed pretty much all its competitors (except for IntelliJ). Java is a strongly typed language and this allows for a lot of analysis that can be done statically and we got used to refactorings and an IDE that fully "understands" our code and its dependencies. Eclipse encouraged us to structure our projects in a way that make dependencies much more explicit by having a set of interdependent projects in the workspace. The plug-in concept enforces a modular structure and is a way of organising the workspace. That's all great.

But there is another reality: all the systems and languages that are much less statically defined and that are structured differently than java. It starts with C. CDT essentially started by copying JDT, it still suffers form that, because C and C++ is a different languages with very different mind-sets than java, although there are some syntactic similarities

Similarly, trying to impose the concepts and mind-set of Java on all those scripting languages and setup-files is probably doomed to fail. I love strongly typed languages, I love modelling. In that area eclipse has so much to offer! But there is the other text based "unstructured" world with a different mind-set, where you "simply add a line to a script to get that cool feature going" (I have to say a lot about frustrations and benefits this "it's really simple" approach can create).

Is eclipse missing this huge opportunity that has opened in the last years with the shift in direction of web technologies?

The "force of intelligence" applied to eclipse

A few weeks ago, I posted a strange and probably hard to understand article about an equation that describes the physical force of intelligence. The equation essentially says, that intelligent systems try to move into directions where they are not trapped. If we apply the equation of intelligence to this problem, we have look at eclipse and at Jetbrains (the creators of IntelliJ) as two distinct "intelligent systems". Both organisations are "intelligent systems" with a "will to survive". The way those organisations act intelligently, is by moving into a direction that "maximises their future freedom of action", as I explained in the blog mentioned above.

In eclipse, nobody has real power to "direct" what is done, instead, the eclipse eco system has to surf the waves that are coming. Its intelligence is a kind of swarm intelligence. And, I think, eclipse has done a pretty good job on this. If the common interest of eclipse (as an "intelligent" system) happens to be in sync with the interest of one or more of its contributors, that is great. But just seeing where eclipse should go, without the power to steer it in that direction, is not "intelligence" according to the equation of intelligence (intelligence requites anticipating possible futures and the power to move into a promising direction) [by that definition this blog entry may not be intelligent, because I have no power to move eclipse].

Jetbrains, on the other hand, has a pretty clear gradient to follow: it has to satisfy its customers in order to make them to pay for the product, which will give them money to maintain and increase their power of action in the future. I assume the JetBrains has more control over its employees than "eclipse" has over the community to drive the development into a "desired" direction. If they continue to make clever moves they may outperform eclipse in the long run.

If I had superpower and could drive eclipse....

What would I do, if I could influence the direction of eclipse and work with a team of people that would have the manpower to make some changes to eclipse?

I think, with a few relative simple changes or additions, eclipse could increase its future freedom of action dramatically by embracing the reality of many developers. The key factors are: 
  • simplify, simplify, simplify
  • install a minimal set of plugins and suggest additions
  • embrace the file system -- don't impose projects on directories
  • don't try to build everything in java -- integrate external tools
  • embrace the command line
  • do clever search and code completion using heuristics instead of complicated models
Or in one sentence: eclipse could be an editor that talks to its environment that can grow to full blown IDE if the user needs and wants it. Instead of being an all-inclusive self-contained IDE that can be abused as editor.

Here are some more concrete ideas to focus on:
  • Create a simple extensible editor with pluggable syntax highlight (something like LiClipse by Fabio Zadrozny)
    • leverage existing (external) syntax highlighting tools to do the job (tools that are not necessarily written in java)
    • or allow to use existing syntax highlighters like the ones form TextMate
  • allow to simply create projects that are whole directory trees including different git/subversion repositories
    • Essentially, allow to point to one one or more root directories and with some clever filtering, allow to exclude parts of the file-tree that are relevant. 
    • Make it simple to assign properties, like 'generated', 'test-code', 'external-library' to subtrees (using patterns or by some UI) to be used for filtering
  • Allow to easily integrate external tools and builders. There is a strong tendency to rewrite tools in java to integrate them. Instead, provide an environment that allows to run, manage and communicate with processes that run externally on the local or on remote machines.
  • Create a clever non-modal search tool, that understands the basic syntax of languages and that can be configured to know where to search, given a context.
    • When I am in file X, I know that file Y is relevant but file Z is not connected to where I am. The problem with search is often how to limit the search scope cleverly. 
    • Typical filters on string searches would be: Do or don't search generated files, test files or external libraries; do or don't search in comments or strings; do or don't search local variables. With some basic understanding of scopes and different areas (which could come for the syntax highlighter), search can be dramatically improved.
  • A plug-in recommender
    • Provide a mechanism that suggests to install  plug-ins based on the content of the project 
  • Provide a generic heuristics based code completion tool (similar to CodeRecommenders). 
  • Heuristics based code navigation. Instead of trying to do full static code analysis of like JDT, use clever heuristics for code navigation and "learn" form users choices.
  • Provide a simple integration with command line debuggers
    • Debugging scripting languages is often much simpler than e.g. C, C++, or hardware debugging
    • It might be enough to provide a simple integration for a command line debugger. (In the 90ies I wrote an extension to the python command-line debugger that made the commands act like gdb, so I could use the gdb integration in emacs and SNiFF+). 
These are just bullet points and if there is interest, I could expand on each of them. The underlying idea is to embrace the text based "unstructured" nature of lots of systems and go back to a simple editor that can progressively be more clever as the user needs it. Do context-based search and navigation and try to syntax highlight as many languages as possible. Instead of being the super-capable IDE that knows everything and contains lots of feature, the user never uses, be the editor that progressively becomes an IDE as the user needs it.

If eclipse cannot be used as a strong editor, the chances are low that we will attract all those developers that simply use and need a capable editor. The world is changing fast, and I wonder how many projects today can ignore the web and mobile apps that use all those dynamically typed "languages" and set-up files.

Promising future for eclipse

I think, eclipse has a bright future! I see some good attempts to go in a promising direction. The risks is not lack of ideas. The risks are lack of resources and money to fund the work that needs to be done. There are a lot of freelancers and small companies, that form the eclipse eco system, making most contributions and working very hard to keep eclipse successful. They all act very intelligently, which means they have to balance their own survival and freedom of future action with the eco system they live in.

Is that enough? How to increase the "power" to act to make eclipse as a whole more "intelligent"? What do you think?


  1. The core of this article can be summed up as better user experience. Companies that based their main IDE on Eclipse, such as Red Hat, should be submitting more resources to improve it. It deeply affects the quality of their offering and their competitive standing.

  2. It looks to me that you are describing what Pascal Rapicault wanted to do with EasyEclipse [1]. I supported the project as much as I could (by funding the Kickstarter project and by speaking about it). It didn’t work. Sadly not enough people are ready to give money to move forward. Companies having invested on eclipse do not seem ready to pay for open-source software. As soon as a company recommends Eclipse as IDE to its developers, I consider this as an investment (think of training and cost to change it).

    Some of the companies using Eclipse IDE become members of the Eclipse foundation. I am not sure they understand the distinction “Eclipse Fundation” vs “Eclipse IDE”. The “Eclipse foundation” was never designed to develop and fix the Eclipse-IDE. Read the eclipse foundation mission statement [2] if you want to be sure. This is not a troll on the Eclipse Fundation. Staff members of the foundation are doing their job well. The foundation is about open-source development, clear IP, process, federating people with common interest… It is not about giving a direction to the development of the IDE. The foundation does not ensure that the IDE will live forever. The foundation is just about making the IDE possible by hosting lots of projects related to the IDE.
    For the moment, nobody is leading the IDE, as a product.

    I appreciate reading blog post like yours. You made a very nice description (your wish list looks like a possible roadmap to me). As you told the problem is “the lack of resources and money”. Someone (a company, a start-up, an additional foundation focusing on the IDE…) that is powerful enough (with enough resources) needs to make the IDE development moving forward in a direction that is useful to a majority of developers.


    1. Jeremie,

      I think you brought up a very good point: many people do not understand the difference between the "Eclipse Foundation" and the "Eclipse IDE".

      The eclipse Foundation is moving in direction of IoT (Internet of Things) which keeps the foundation relevant and alive. IoT seems to have a very bright future and surfing that wave is a great idea for the foundation. (Also in therms of the equation of intelligence)

      But success IoT may or may not help the Eclipse IDE. It could well be that if a few years form now, the Eclipse Foundation is very successful in IoT (or whatever is the focus of the Foundation at that time), but the Eclipse IDE is starving....

  3. "The risks is not lack of ideas. The risks are lack of resources and money to fund the work that needs to be done." - I can't agree more. Working in the sphere of mobile app development I face the situation very often.

  4. no matter how good the automation in other commercial IDE such as intellij, there is something about eclipse