The Mysterious Case Of The Lost Inherited Call

Soon after I released the free version of MMX Code Explorer, I got complaints from people having problems after they tried it in Delphi 10.2 Tokyo initial release and with Update 1 installed. The version I released was compiled with Delphi 10.2 Update 3 and when loaded in a Delphi 10.2 with no update or only Update 1 it popped up with an error message about a missing procedure entry point “@Dockform@TDockableForm@DoEndDock”.

As a quick fix to this problem I released another version compiled with Delphi 10.2 Tokyo (no updates), which solved it for those versions as well as apparently made no harm to installations with Update 2 and 3  – apparently!

After I got the complaining users satisfied for the moment I investigated a bit more closely what the real cause of the problem actually was. Despite seeing no direct problems with the workaround above, I somehow felt a bit uncomfortable. Perhaps there was some little beast lurking in the dark, which may raise its ugly head some day when no one expects it. In addition, having to compile with a not up-to-date version doesn’t make life easier on the long run, too.

A straight forward search revealed one class in the code inherited from TDockableForm overriding DoEndDock. As expected there was a call to inherited DoEndDock in the implementation. All looked quite reasonable to me.

Then I moved the mouse over the inherited DoEndDock and waited for the Help Insight window to popup. This is what it looked with the initial Tokyo release:

We can see that the inherited call goes to TControl.DoEndDock. Still nothing to worry about here.

Now lets see what the same code reveals in Update 3:

Hey, that looks indeed a bit suspicious. In Update 3 the inherited call will go to TDockableForm.DoEndDock instead of TControl.DoEndDock.

Now comes the tricky part: Virtual method calling is a one way process!

When a virtual method is called, the most significant override of this method is called through the Virtual Method Table (VMT) – we know that, don’t we? Thus we don’t have to know the concrete class when we make that call. The VMT takes care of the correct method to be called.

Virtual method calling is a one way process!

On the other hand, when we call inherited in this overridden method, the compiler knows, which inherited method of which class is going to be called and resolves that call to the correct address. Seems legit – well, somehow, but yes.

Everything works as expected when we compile a single executable where all methods are linked in and entry points are known. Things look different when packages come into play.

When compiling with packages the compiler uses the DCP file to learn which method has to be called with inherited. At runtime the corresponding BPL will then provide  the proper procedure entry point. Unfortunately this mechanism fails, when the runtime package version is different. In that case the procedure entry points don’t match or are simply non-existent.

In this case Update 2 introduces an override of TDockableForm.DoEndDock adding a new procedure entry point. A compiler seeing this override will rightly call this method instead of TControl.DoEndDock. Now when a package before Update 2 is used during runtime, we cannot find that entry point any more. That is exactly the error we got in the beginning.

So, does compiling the expert against the older DCP really solve the problem? At least it makes the error message go away when run in the older environment. But what about running under Update 2 or 3? The method called for inherited is still TControl.DoEndDock and that exists and the call succeeds. The bad thing about this is that we miss to call TDockableForm.DoEndDock in this case. I have no idea what this method does, but with the current version of the expert it is simply never called for all dockforms created by the expert. I could feel a deep grumbling in my stomach when I realized that.

The conclusion to be drawn from this is: When it comes to packages and you want to be binary compatible between your package versions, you better not touch the interface sections of your units! Introducing a new inheritance level for a virtual method – or worse, removing one – will break binary compatibility. That is a direct consequence of how the compiler resolves inherited calls.

For Delphi this limits the possibilities of what can be implemented in a so called point release, where binary compatibility is key.

How can we overcome this limitation? It depends. In this case I removed the overridden method and wired an event handler for OnEndDock, hoping that nowhere in the IDE some other event handler will steel this link. I know that this will not always be an option.

The next release will incorporate this fix, which also allows me to compile with Delphi 10.2 Update 3 again. Simplifying the build process is currently my main task regarding MMX Code Explorer. The need to use a specific Tokyo version for compiling the expert threw a massive burden on my build system. Having things back to normal is a big relief.

 

 

The only new feature in MMX

The move of MMX turned out to be very welcomed in the Delphi community. Over 3000 downloads during the first 48 hours are just overwhelming and unfortunately led to some interim drop outs of the web server. Apologies for that. I will talk to the web hoster about this and see if we can get this a bit more stable in the future.

The original plan was to release with the exact same feature set as the then current version of MMX with just the licensing removed and all the references to ModelMaker Tools BV rewired to the new home. Well, that didn’t work out exactly like that. There is one new little feature in V13 that somehow slipped through. I doubt that many of you long time users found it unless you actually scrutinized the option dialog, so let me explain what it is and what it does.

This is a screenshot of the Pascal Sorting options dialog with the checkbox Group and sort uses highlighted. With this option checked any Format Uses Clause action (default Ctrl-Alt-U) will resolve unit aliases and expand unit scope names. Then units are grouped by scope names and sorted by group in the order given by the projects options scope names entry. For example:

will be changed to

As with the original uses clause formatting this works only when there are no conditional defines inside the uses clause.

I implemented this feature more as a exercise to become familiar with the code and somehow forgot to exclude it from the release. It also is not fully complete, because the only way to influence the grouping and order is to adjust the unit scope names of the project, where a project specific settings file (or a new entry in the dproj file) would be more appropriate. So I guess this will be completed in one of the next releases.

Now as it is public I hope it might be useful for some of you.

A New Home for ModelMaker Code Explorer™

I am very pleased to announce that as of today I am going to take over the award winning ModelMaker Code Explorer™ from ModeMaker Tools BV.

For over two decades ModelMaker Code Explorer™ is still an indispensable add-on to the Delphi IDE for numerous developers. Speaking for myself, a Delphi installation is simply incomplete without MMX. The benefit in using it always outweighed the price of this tool in magnitudes.

With ModelMaker Tools BV concentrating on its Mind Mapping tool SimpleMind, further development and distribution of MMX is now completely in my hands. To emphasize the movement away from ModelMaker Tools BV there will be a slight change in the name: ModelMaker Code Explorer™ will become MMX Code Explorer from now on, still shortened as MMX. The new website can be found here: https://www.mmx-delphi.de/

As a first and probably widely welcomed benefit MMX will be available for free with immediate effect. This allows even all hobbyist developers with a tight budget to make use of this tool. MMX will nevertheless stay closed source, though.

The first free release 13.0 is more or less the same as the current commercial release 12.3 in regard to functionality with all the licensing removed and (hopefully) all references to ModelMaker Tools BV changed to new targets. Alas, the Delphi 5 and 6 versions didn’t make it into this transition, as they would have required too much special treatment. The current release thus supports Delphi 7, Delphi 2007 – 2010, Delphi XE – XE8, Delphi 10 Seattle, Delphi 10.1 Berlin and Delphi 10.2 Tokyo.

My plans for the coming releases are mostly:

  • cleaning up the code base (f.i. remove all C# related code)
  • restructuring the code while removing dependencies where possible
    (we all have our own ideas when it comes to architecture, don’t we?)
  • updating the documentation and examples (videos)
  • supporting new Delphi releases (goal is from day one on if not even beta)
  • better support of newer language features
  • implementing some ideas I collected over the last years

I am not going to conceal that this will most likely lead to drop support for older Delphi versions in favor of making maintenance easier. The latest MMX supporting those retired versions will nevertheless be available for download even after that.

Finally I want to say a big THANK YOU to Gerrit Beuze from ModelMaker Tools BV for his generosity and invaluable support in the past and during this transition.

Will MMX live long and prosper!

Keep Your Project Files Clean With Project Magician

This is sitting on my desktop for quite a while waiting that I find the time to write this blog post. So now be it.

In the past a couple of you have downloaded and probably also used my DprojNormalizer IDE plugin. Well, I have to say that I am not going to update that plugin any longer. But don’t hesitate – a replacement product will  be available with some interesting features added: Project Magician

DprojNormalizer as well as DprojSplitter are one-purpose plugins. If they are installed, they do their jobs. No switches, no settings inside the IDE, although DprojSplitter respects an optional configuration file. Both plugins can be installed side by side without one interfering with the other.

Although this lean approach has some advantages, it also has some drawbacks: a good portion of the code is similar and the IDE events handled are almost the same. But the most prominent issue I have with these one-purpose plugins is: I am running out of proper icons.

This was the birth of Project Magician which combines the features of DprojNormalizer and DprojSplitter adding some new features that roughly fall into the same area.

In contrast to both other plugins Project Magician now has settings inside the IDE. Have a look at the settings form for the Project Magician project itself.

The settings are grouped in four different areas.

Project file settings contains the Normalize and Split entries, which resemble the already know behavior of DprojNormalizer and DprojSplitter. The third option in this group allows to remove the Excluded Packages entries from the project file. Usually this entry is unnecessary and depends on the local installation often causing trouble on commits. It will be re-created automatically with default settings when a project is opened.

Clear Settings in Child Configurations probably needs some words of explanation. Sometimes the inheritance of project settings as implemented in the IDE is not wanted and leads to a lot of work or unexpected results. For instance, having different version info values for each build configuration is rarely required. Changes made to one configuration are not visible in others. In addition inheritance doesn’t work well with these either. Changes in the base configuration have no effect if values in depending configurations exist.

That said, the Version info option clears all settings made in child configurations and only keeps the settings in the base configuration. In case of a pure Windows project this actually is the base configuration, while for mixed OS projects each platforms base configuration serves as a basis. Note that the Module attributes are excluded from the clearing so that each build configuration can have its own attributes.

Application Settings are always platform specific and thus the settings in the platform base configuration are the ones kept. The settings targeted here are Icons, Manifest File, Output Settings and Appearance.

Package Settings affect Prefix, Suffix and Version as well as the Description and the Never Build option. Needless to say that Package Settings are only enabled for packages while Application Settings have only a meaning for applications.

When upgrading a project from a previous Delphi version it sometimes happens that you cannot add platforms even if they are available in the IDE. This is where Enable Missing Platforms steps in, enabling all platforms known to the current Delphi version, even if they are currently not available for your SKU. Although you still can only add those platforms provided by your IDE, the other platforms can later be enabled on systems that include them.

Projects that are only useful for some platform and will never be used on others (like f.i. design time packages) are nevertheless cluttered with platform specific entries for these never be used platforms. When the Remove Unused Platforms option is set Project Magician will remove those entries resulting in a much cleaner project file. This doesn’t remove the ability to add those platforms to the project later.

Maintaining packages for different Delphi versions requires to use a decent LIBSUFFIX to distinguish the bpl-files for each version (see Delphi Library Guidelines). Creating the project files for a new Delphi version usually involves copying those from the previous version to a new folder and adjusting the LIBSUFFIX entries. The Auto LibSuffix option eliminates the last step.

Sometimes the FormType entries in the project file are missing. This leads to the fact that during design time one cannot instantiate frames affected by this. The  Refresh Form Type functionality restores the missing entries when opening a project.

As you may notice there are settings for different scopes: global, project group and project. Settings are inherited from the next higher scope and can only be activated in a lower scope, but not deactivated. Take care when activating settings on global level! In the example above Refresh Form Type and Remove Unused Platforms are set on project group level.

The global settings can alternatively set from the Tools – Options dialog. The Project Manager options can be found in the Third Party branch.

Only as part of the global options you have access to the Clean Line Feeds feature. When opening dpr, dpk, pas or inc files any invalid line feed sequence (like single <CR> or <LF> alone) will be converted to a proper <CR><LF> sequence.

Project Magician comes with a command line application that allows to execute most of the tasks done by the plugin from the command line. It is named ProjectMagicianCmd.exe and resides in the installation folder (default: %ProgramFiles(x86)%\ProjectMagician).

The general usage is:
ProjectMagicianCmd [-v:<version> | -n | -r | -x | -f] [<filepath>]<filename> [-l:<logfile>] [-s]

The parameters have the following meaning:
-v = Sets VersionInfo in dproj files to a given value. Clears all version info entries in child build configurations.
<version> up to 4 numbers separated by dots

-n = Normalize

-r = Removes unused platforms

-x = Removes “Excluded Packages”

-f = Refreshes and adds missing form type entries

<filename> may contain wildcards. If no extension is given .dproj is assumed

Project Magician can be downloaded here:  Project Magician

Currently Delphi XE3 to 10.2 Tokyo are supported.