Surprises in eclipse 3.3: double click instead of single click

Since some time, the 3.3 patch editor (Team->Apply patch...) does not work for me anymore. Well, I was using patched milestones (we had some enhancements for the patch editor in our product). So, I was not surprised that it did not 100% work. What went wrong? If I select a file to patch, nothing is shown in the compare area below:

Since the problem remained even with an unpatched 3.3m7, I filed bug 186481 and after some discussions, it turned out that I have to double click instead of single click to select the patch. That was not obvious to me! In 3.2 a single click was enough! After double clicking the patch it looks like this:

Now I see exactly what I expected! But why double click? Well, because calculating the diff is expensive and doing that by simply selecting a file may not be a good idea... Another reason is that there's now a context menu on the items in the "Patch Contents" pane and you don't want to do the expensive calculation of the diff just to get to the context menu. Therefore, the paradigm changed from master-detail to open semantics.

But this is not without drawbacks! First of all, I'm sure I'm not the only one stumbling over this problem. Actually the semantics has changed in other compare related views too. If you set Preferences...->General->Single Click, you get the old master-detail behaviour.

Second drawback, is that the selection and what you see easily may get out of sync. Look at this screenshot:

What happened? I have selected (not double clicked) the plugin.properties file but the difference of a java file is shown. The bad thing is there is no indication which java files is shown here! Unfortunately it's too late in the game to change this (see bug 186481 for further details)...

Take-home message: Report bugs early enough....

No real support for dynamic extensions (aka dynamic plugin.xml)

Eclipse allows (theoretically) to add extensions dynamically (see "Leave Eclipse plug-in headaches behind" or "Contributing items to a toolbar dynamically"). Unfortunately, there's no way a normal plugin can do this without using internal classes :-(. Here's the proposed hack how to do it:

  IExtensionRegistry reg = RegistryFactory.getRegistry();
  // ExtensionRegistry is internal!!!!
  Object key = ((ExtensionRegistry) reg).getTemporaryUserToken();
  Bundle bundle = Activator.getDefault().getBundle();
  IContributor contributor = ContributorFactoryOSGi.createContributor(bundle);
  try {
   //  I have the content of my dynamic plugin in a file 
   // called dynamicplugin.xml
   InputStream is = FileLocator.openStream(bundle,new Path("dynamicplugin.xml"), false);
   reg.addContribution(is, contributor, false, null, null, key);
  } catch (IOException e) {
  }

Unfortunately ExtensionRegistry is an internal class and getTemporaryUserToken() is not exposed in any official way. Bug 112954 asks for a way to get to getTemporaryUserToken(). The solution is to use null as the key (token). Well, almost. In order to make this work, the system property eclipse.registry.nulltoken has to be set to true (Bug 112954 comment 25). But a normal plugin cannot set a system property before OSGi starts....

This effectively means: there is no way a normal well behaving plugin can provide a dynamic extension. Therefore I filed bug 174967 to fix that problem....

java.net.URL.equals and hashCode make (blocking) Internet connections....

Sometimes simple calls have unexpected side effects. I wanted to update some plugins, but the update manager was hanging my UI. Looking at the stack trace reveals:

at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
at java.net.InetAddress$1.lookupAllHostAddr(Unknown Source)
at java.net.InetAddress.getAddressFromNameService(Unknown Source)
at java.net.InetAddress.getAllByName0(Unknown Source)
at java.net.InetAddress.getAllByName0(Unknown Source)
at java.net.InetAddress.getAllByName(Unknown Source)
at java.net.InetAddress.getByName(Unknown Source)
at java.net.URLStreamHandler.getHostAddress(Unknown Source)
- locked <0x15ce1280> (a sun.net.www.protocol.http.Handler)
at java.net.URLStreamHandler.hashCode(Unknown Source)
at java.net.URL.hashCode(Unknown Source)
- locked <0x1a3100d0> (a java.net.URL)


Hmm, I must say that it is very dangerous that java.net.URL.hashCode (and URL.equals) makes an Internet connection. java.net.URL has the worst equals/hasCode implementation I have ever seen: equality depends on the state of the Internet. Well in the javadoc of URL.equals it says: "Since hosts comparison requires name resolution, this operation is a blocking operation.", but who reads the documentation of equals? There is a general contract around equals. Joshua Bloch writes in Effective Java: "Don't write an equals that relies on unreliable resources" (Chapter 3, page 34). Hey Sun, as far as I know, the Internet is not reliable ;-)

Do not put java.net.URL into collections unless you can live with the fact that comparing makes calls to the Internet. Use java.net.URI instead.

URL is an aggressive beast that can slow down and hang your application by making unexpected network traffic.....


I wonder if other people find this behaviour as shocking as I do....

Tip: debug (and patch at runtime) an installed eclipse

Today I had a problem that occurred only in my installed eclipse but not in my runtime workbench.

How to start eclipse so you can attach a debugger?

You have to add the following to the vmargs: -vmargs -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=9009
The 9009 is used later to attach to eclipse.

How attach to eclipse?

• create a new Debug->Remote Java Application.
  
• Set the port to to 9009 (or whatever you use as the address argument in your eclipse launch)
  
• On the Sources tab add the projects that you want to debug..
  

How to "patch" the running eclipse?

Just modify the code. Actually, I discovered this by accident. My problem was an obvious bug (missing check for null), so I fixed in during the debug session. And the usual eclipse "magic" fixed the bug in my running eclipse. It's the hot code replacement that normally happens. I (falsely) though that would only work when I start a runtime eclipse from within eclipse. But it works in any case.

Now I can fix my running eclipse while I'm normally working. The only downside is, that the two eclipse must use two different workspaces. Too bad.

Here's a flash demo video that shows how to do this step by step.

Help! How to manage multiple (RCP) applications in one installation?

Maybe someone reading my blog can help me. I posted essentially the message below to the equinox newsgroup and the platform newsgroup with zero response. Maybe RCP in the headline will help getting someone to read my post ;-)....

How to restrict the plugins visible to an application?

We have several applications (some of them are headless) that run from within the same eclipse installation. The problem is, that all plugins are visible from the applications. This cause problems, if extensions are used that load plugins that cannot run in the context of the headless application.

To illustrate the problem I use a simple example. A headless application that uses the org.eclipse.core.variables plugin to expand a variable. The org.eclipse.core.variables is UI free and therefore no problem. But some plugins that contribute to the variables plugin are not headless. To illustrate this, I use the ${selected_text} variable that is defined in the org.eclipse.debug.ui plugin. This causes the application to fail.

  public class MyApplicationWithVariables implements IPlatformRunnable {
    public Object run(Object arg) throws Exception {
      IStringVariableManager vm=VariablesPlugin.getDefault().getStringVariableManager();
      String message=vm.performStringSubstitution("Hello ${selected_text}!");
      System.out.println(message);
      return EXIT_OK;
    }
  }

   <extension
         id="HelloWorldWithVariables"
         name="Minimal Headless with variables"
         point="org.eclipse.core.runtime.applications">
      <application>
         <run class="gr.scharf.minimal.headless.MyApplicationWithVariables"/>
      </application>
   </extension>

Is there a way to restrict the plugins visible or loadable to an application?
Or is the only way to have a copy of the installation containing only the "good" plugins?

Michael

How the eclipse update manager could look like...

One of the features I don't like about eclipse is the update manager. A few weeks ago, I looked at different alternatives of eclipse update managers. Today Chris Aniszczyk blogged about the FireFox update manager. Interestingly, a few days ago, I created a small flash video on the FireFox update manager, because I think that shows what a user expects from a simple to use update manager.

One of the central paradigms of eclipse the idea of plugins. However from a user perspective, managing plugins is a nightmare. The eclipse update manager is way too complicated, too slow, not robust enough.

Few weeks ago I bought Eric Evans book "Domain Driven Design". I'll blog about it separately. One of the central ideas is what he calls "ubiquitous language" (e.g. read chapter one). The basic idea is to come up with a language to describe the problem domain, in our case the installation and update of plugins. The language is not only used in conversations but also represented by a formal model. The formal model is implemented in code. You can start reasoning and discuss about the model.

If I look at the eclipse update manager, the model used is a very technical model that is totally implementation centric. It is not a model a user of the update manager can simply understand. I think this is the root of the problem. We have to come up with an update manager domain model that represents the problems and tasks a user of eclipse faces and not a model that is focused on the implementation. The current implementation could be used to implement the user model, but in it self it seem not the right abstraction for users.

The FireFox update manager has an easy to understand domain model that is shown in the UI. There are extensions. An extension has an icon, a description and a set of preferences (yes the preferences of each extensions is available directly from the "Extensions" dialog. In eclipse there's no way to find out what an extension contributes to eclipse. See my blog entry "When seamless integration becomes a nightmare..."). Anyway, hit a simple button to check for updates. It's done in seconds (not minutes like eclipse). Once downloaded, you can update the plugins you want to update.

If you don't like an extension you can select an extension and "Uninstall" it. Something that is not possible with the eclipse update manager. Eclipse is a grow only system -- no way to remove features. Like windows in the old days. Once you have installed a feature there's no way to get rid of it. Only, if you regularly do backups (like I do), you can revert a previous version of eclipse by reverting a backup. But eclipse itself is grow only (*). If you have installed a few features you don't like and you cannot get rid of, you will start thinking twice if you want to install a new plugin. That's very bad for the eclipse ecosystem.

Eclipse must solve the update manager problem soon and provide a simple to use and understandable update manager that solves the problems of eclipse users.

Domain Driven Design could help come up with a update manager domain model that is relevant for eclipse users...

Michael

(*) You could also use separate extension locations for each feature you install and simply delete the location if you want to uninstall the extension. But that's a hack. The update manager (UI) does not even support dropping of extensions. I tried that for a while, but it requires a lot of discipline.

When a system becomes corrupt....

Have you ever had a system running stable for years and suddenly everything goes wrong?

It started last Saturday, when one of my usb-backup drives died. No big deal. I have two external backup drives with at most one attached to my system. I do weekly full backups and daily incremental backups on important data. I swap drives from time to time (I should swap them every day) and keep some old backups.

On Monday my DSL connection died. No problem, I have a ISDN USB modem I can use in those cases. The modem was cheap but the driver sucks: it uses 100% CPU even if nothing is going on. Well, no problem, I downloaded a new driver, created an XP system restore point and installed the driver. Well, I tried different versions of the driver. No one really helped, the last one crashed my system. OK, I did a "system restore", but during shutdown the driver crashed my computer. Restart, login ... baang! Windows forgot my user settings! It created a new profile. All settings lost!

No problem, I first create a backup (it's always a good idea to backup the entire system, even if is corrupt, before you mess around!) and then I wanted to back the last full backup...

Big problem! The last backup is corrupt, and the most recent backups are on my dead drive...

...for the last 12 hours I have been playing back different versions of backup trying to restore my user data. In between I have between googling for the potential problem of my setup, with little success. Whenever I restored my latest user data in C:\Documents and Settings\<username>, windows decided to create a new profile for me. I got very very frustrated and finally accepted the fact that I have to live with a new profile and I have to set up my environment from scratch. I copied all my user data into the new profile....

...Firefox, Thunderbird, cygwin, eclipse are applications that did not loose data and were running as before. But my desktop, internet explorer and many other applications that depend on the registry lost their settings. The worst case was Outlook, it wants to be reinstalled and looses all setting that are not on the exchange server...

So, I asked myself, is there a way to get my personal registry settings into my new profile? Finally I stumbled over the important files: the C:\Documents and Settings\<username>\ntuser.* files. From my "after the problem" backup I figured out that the file ntuser.dat file was missing.

Was that the key to my problem? Did loosing the ntuser.dat cause all the troubles? Ok, I had to try it. I copied an old version from a backup into my corrupt profile. Hmm, but how to tell XP to use my old profile directory again? A quick search in the registry revealed that the profile directory is stored under HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\ProfileList/*. I changed it, renamed my new profile to something else and restarted....

WOW! It works. My system is back. Outlook works, my desktop looks like always, all applications work as expected. I'm happy again....

What has my windows problem to do with eclipse?

Eclipse does not use the registry, but it has the magic .metadata directory. Over the last 5 years my .metadata directory got corrupt several times. If something goes wrong with the ..metadata (like running out of disk-space), it's very hard, if not impossible, to recover from that. You can, similar to windows, create a new workspace and import the projects. But then you have lost a lot of settings and you "start from scratch". My backups saved me several times here.

Conclusions

• Backup the corrupted system before you do anything.
  
• Make bakups. Regularly!

Michael

Mylar is very cool, however sometimes it drives me nuts...

Mylar is an eclipse technology project. It makes it easy to focus on a task you are working on, by hiding the information that's not relevant to the current task.

How does it know, which information is relevant? It "looks over your shoulder" and remembers the files, classes and methods you have looked at. With some magic it puts some weight on the places you have been. For example if you get back to some place or even edit there, it gets more weight than just looking once at the file. But you can also tell mylar explicitly that you are interested or not interested in some context.

How does it hide irrelevant information? Well, in the package explorer, outline etc you see only the classes and methods in your context. There's even a mode that shows the relevance by the background color. And really important places are shown in bold. To build up context, you can disable mylar on the view to see everything. I never use the editor folding features of mylar, because I don't like folding.

If you want to learn more, I highly recommend the getting started flash demon.

There are also integrations with bug tracking systems, and it seems that this is how most people are using Mylar. Unfortunately, we use a commercial bug-tracking system that is not supported by mylar. But I find mylar very useful, even without the bug tracking system integration.

However, sometimes I hate mylar: I am working on a problem, and I find something interesting I want to look at later. So, I create a new mylar task. That's all fine. But now, when I want to add context to this task, I have to start from scratch, because mylar closes all my editors and collapses all my trees. However, if I have switched off mylar (that is: I have no active mylar task), and then create a new task and activate that new task, everything is good. In this case mylar does not close all my editors, but is clever it letting me build up my context from where I am. (I created a little flash demo to illustrate this workflow). I talked about this on the newsgroup, and it seems that my workflow is very special. I wonder if I am the only one having this workflow problem with mylar, therefore I created a bugzilla entry: bug 157933....

Don't swallow InterruptedException. Call Thread.currentThread().interrupt() instead.

Have you ever written the following code?

try {
    doSomething();
} catch(InterruptedException swallowed) {
    // BAD BAD PRACTICE, TO IGNORE THIS EXCEPTION
    // just logging is also not a useful option here....
}

I have! I newer knew what the heck to do with those annoying InterruptedException when I simply wanted to call Thread.sleep(..). What is this InterruptedException? Why is it thrown? Why can't I ignore it?

Before I explain the whys, here is what you should do (if you don't re-throw):

try {
    doSomething();
} catch(InterruptedException e) {
    // Restore the interrupted status
    Thread.currentThread().interrupt();
}

What is the InterruptedException?

There is no way to simply stop a running thread in java (don't even consider using the deprecated method stop()). Stopping threads is cooperative in java. Calling Thread.interrupt() is a way to tell the thread to stop what it is doing. If the thread is in a blocking call, the blocking call will throw an InterruptedException, otherwise the interrupted flag of the tread will be set. A Thread or a Runnable that is interruptable should check from time to time Thread.currentThread().isInterrupted(). If it returns true, cleanup and return.

Why is it InterruptedException thrown?

The problem is that blocking calls like sleep() and wait(), can take very long till the check can be done. Therefore they throw an InterruptedException. However the isInterrupted is cleared when the InterruptedException is thrown! (I have some vague idea why this is the case, but for whatever reason this is done, that is how it is!)

Why can't InterruptedException be simply ignored?

It should be clear by now: because ignoring an InterruptedException means resetting the interrupted status of the thread. For example, worker threads take runnable from a queue and execute they may check the interrupted status periodically. If you swallow it the thread would not know that it was interrupted and would happily continue to run.

Some more thoughts
Unfortunately it is not specified that Thread.interrupt() can only be used for cancellation. It can be used for anything that requires to set a flag on a thread. So, ending your task or runnable early might be the wrong choice, if the interrupted status is used for something else. But common practice is to use it for cancellation. But even if it is used for something else, you code does not have the right to reset the interrupt flag (unless you are the owner of the thread).

To learn more read the nice article Dealing with InterruptedException by Brian Goetz.

Or read Java Concurrency in Practice By BriaBrian Goetz, Tim Peierls, Joshua Bloch, Joseph Bowbeer, David Holmes, and Doug Lea. It's a great book! If you program in java 5 this book is a must read!. I bought the book after I read Brians artikle. I have not yet read the entire book, but what I have read impresses me. It gives a lot of theorie and options. Very competent and complete, but I'm missing a set of simple patterns for concurrent programming. I'm looking forward to Doug Leas 3rd edition of Concurrent Programming in Java

Summary: If you don't know what to do with an InterruptedException call Thread.currentThread().interrupt()

Pitfalls of Long.parseLong/Long.decode...

What is wrong with this code?

assert 0x0cc175b9c0f1b6a8L == Long.parseLong("0cc175b9c0f1b6a8",16);
assert 0xd41d8cd98f00b204L == Long.parseLong("d41d8cd98f00b204",16);

The second Long.parseLong will fail in the second line:

java.lang.NumberFormatException: For input string: "d41d8cd98f00b204"

The reason is that javas long is signed, but 0xd41d8cd98f00b204L is an unsigned long. Interesting, but annoying, if you want to parse an unsigned long in java. It makes Parsing and verifying hex, octal and decimal numbers more complicated, if you have a field that represents a C/C++ unsigned long (a 64 bit number)....

The solution is to use BigInteger:

assert 0x0cc175b9c0f1b6a8L == new BigInteger("0cc175b9c0f1b6a8",16).longValue();
assert 0xd41d8cd98f00b204L == new BigInteger("d41d8cd98f00b204",16).longValue();

However, you must ensure that the BigInteger does not exceed 64 bits. So here's my unsigned long parser

public static long parseUnsignedLong(String s, int radix) 
    throws NumberFormatException {
    BigInteger b= new BigInteger(s,radix);
    if(b.bitLength()>64)
        throw new NumberFormatException(s+" is to big!");
    return b.longValue();
}

Note: if you want to do any kind arithmetic with unsigned longs (like min/max checking) you have to use BigInteger objects.... Unless you are only interested in the bits of the long, the return value of my parseUnsignedLong is not really useful: it should better return the BigInteger!