Object libraries, again

A few weeks ago I made a post about the problems that arose with not so experienced EZ-Scenery users trying to change/alter object libraries. In the past few days I come across a few more discussions that relate to this and what I read there worried me a lot. So therefore this new post about the subject.

What worries me the most is that some people think about (or maybe even put into practice) the idea of decompiling libraries, to then put the objects back into new libraries that are organized according to their preference. So for example all hangars together.

Having a way to organise your objects is not bad of course, that is a very useful feature. But you should NEVER alter existing libraries during that process. There are quick a few good reasons for this.

  • When you create your new library, all objects will get a different identification number (GUID).  This is certainly not prefered. Think of the chaos that would appear when we end up with one object having dozens of different GUIDs in the end, because a lot of users made their own personalized library. You would never know from which library you are reading that object at the moment.
    It is also just against the reason of libraries in the first place. There are meant to be as a collection of objects, that can then easily be used in different sceneries.
  • In the end if would cost you a lot of disk space as well (and thus more downloading for the users of the scenery as well). Would you be happy if you had to download the same objects again and again? Of course not, it is much easier to download one library containing them all once and after that any scenery can make use of them.
  • By changing the GUID that the origional author has assigned you also break his backwards compatibility. Imagine that he uploads an improved version of his library. Any user that uses his origional one wouldn’t have to change anything, their placements refer to the GUIDs of the author and they can thus automatically use his new objects. But if you have assigned a new GUID to the object, you loose this.
  • When compiling the work of somebody else, you are usually breaking his copyright/user license. So you could get in trouble if you distribute this new library.

I am not saying that those EZ-Scenery users that come up with these ideas are to blame here. They probably just miss a bit of “education” about what an object library is and how you can use it. So maybe it would be worthwhile to put some effort into that. Maybe it should even be part of the documentation of object placement tools, especially if they aim at a broad public that does not only contain hardcode scenery designers.

I also want to give some attention to another issue that came up in recent discussions. EZ-Scenery does scan your entire scenery library for object libraries. And they are then all added to the list you can choose from. But what happens if you have an addon scenery installed that also makes use of an object library? It will also appear in the list of course. If that library has been designed for one specific airport that is not something that you want and if the library is part of a commercial scenery you don’t want it either. In both cases you could never distribute your scenery if it uses objects from such libraries. Simply because you can not distribute the libraries with it.

So I think it is better if object placement tools would only show objects from generic libraries that are part of FS and object libraries that have been designed and distributed with the purpose of being used as a general object library. For example all those libraries that have been created for use with Rwy12.

For the moment we should trust the user of EZ-Scenery and hope he has enough knowledge to make the shift between those generic libraries he can use and the specific or commercial libraries he can’t use. But as they all appear in his list, I am afraid this will go wrong some day.

That brings me to a last point. There is absolutely no difference between an object library designed for Rwy12, EZ-Scenery or ObPlacer XML. The only difference is how those tools read their information about the library. So wouldn’t it be nice if we could get some sort of collaboration between users of all those tools. I would hate to see more Rwy12 libraries or EZ-Scenery libraries appear at the big download sites. We should see object libraries for Fs2004. And it should not matter which tool you use in the end to place the objects.

SCASM is bad and GMax is good, or is it?

Ever since GMax was included with Fs2002 discussion like SCASM is bad and GMax is good have appeared now and then. Now that FsX has been announced I have noticed that I appeared again. I saw questions of people wondering if SCASM BGL files would still work.

But people that ask such questions don’t seem to understand what SCASM is. It is just a compiler that creates a BGL file from a source code with a list of commands that you provide to it. The file that is produced is not something special that only SCASM could make, any other compiler like FreeSC or BGLC can make exactly the same BGL file. So to ask if a SCASM BGL file would still work is nonsense, as there is no way to tell which compiler created the BGL file.

Most people that this question are worried about the support of older scenery commands that they still use. And this is a valid question, but it should not be linked to the compiler used. If the old Fs5/Fs98/Fs2000 commands will still work in the future is something that is a good point of discussion. The introduction of the floating point commands in Fs2002 for example, makes the integer point commands used before almost useless. So it is logical to assume that support for them will be dropped somewhere in the future.

To return to the SCASM question, it is very well possible to make a BGL file that is identical to a BGL file created with the Fs2002 gamepack for GMax. So in that case SCASM is just as good as GMax. For Fs2004 this is no longer valid, as SCASM has not fully been updated to the Fs2004 format. SCASM is able to create mesh elements (VTP/LWM polygons for example) and do object placement (like we can do with BGLComp as well), but it is not able to create the MDL scenery object files. This is because the structure of these files would require quite a big change in the internal workings of SCASM and it is rather unlikely that this change will be made. So in the future we might see that SCASM is no longer equal to the other compilers we can use and maybe then we can start asking if SCASM is “bad”. But not because it is SCASM, because it does not support the latest formats.

New York

I am already a few days back from my trip to New York and here is a little travel report. Unfortunately I also brought back a bit of flu, so that is why I have not been really active the last days.

Our trip started at Amsterdam Airport Schiphol and luckily we departed from the G-pier, so I had a good look at the recently opened H-pier (I just modeled that one, so I wanted to check if it looked right).

The flight itself was with Delta Airlines in a Boeing 767-300. Luckily I had a window seat on this flight, so I could take a good look at the scenery. I have for example been looking at the sun reflection in the ocean for some while. Really cool how all those different colors blend. I must say that the sun reflections in MSFS don’t look that bad, they certainly look better then the ones (or should I say lack of ones) in the image generator we use at work. I guess that is something we can improve in the future.

When we arrived in New York there was an overcast, so unfortunately we could not really enjoy that view from the aircraft. But we had two days left to explore the city, before our course would start. As the city is very big of course, we mainly walked around it to feel the atmosphere. Our hotel was nicely located in Manhattan, so one day we walked to the south till we reached the end of the island. And the other day we went up north  to Central Park. And the last day we walked over the bridges into Brooklyn and back again. The view from those bridges was really nice.

I was impressed by the skyline as well. I have read the book “The island at the Center of the World” from Russell Shorto. It is about the first few (Dutch) years of the city of New York (then called New Amsterdam). In it are also some drawings of the skyline back then. The highest building was probably the windmill they had constructed. If you know see all those huge skyscrappers at the same place, it makes it hard to image how it once was.

Now that I am talking about the Dutch origin of the city anyway, did you know that Brooklyn comes from the Dutch town called Breukelen? And similar to that the name Harlem comes from the Dutch city Haarlem? Those are a few things that still remind of the Dutch origin of the city. Another is Wall Street. At the location of this street, the city wall used to be in the Dutch times. But I guess that is enough history for today!

From New York city we drove by car to Binghamton. Our course was given
there at the university. At first I had never heard of that place and
it did not really make sense why we had to go especially to there for a
course on flight simulation.

But it seemed that Edwin Link
designed and constructed his first link trainer in that area. At the
local airport there is even an model displayed. So this region really
has the roots of flight simulation and that makes it easier to
understand why exactly that university has collected quite some
knowledge on the subject.

The course itself was very interesting. A lot of different aspects of
simulation where discussed. It ranged from motion systems, to visual
systems, from control loaders to image generators. And of course the
aspects of mathematical modelling for the simulation were also touched.
In the end it was a really interesting week that provided a lot of
information about simulation and all its aspects. I will certainly not
become an expert in all, but it should always help to talk better with
the colleagues who are.

Away…

Next week I will be attending the Flight and Ground Simulation Update 2006 at the university of Binghampton (NY). I am really looking forward to this course, that I will attend for my work. While I am on the other side of the Atlantic I am also going to spend two weekends over there as a sort of short holiday (going to visit New York for example). So don’t expect any new blog posts during that period.

Once I am back, I am sure that I have some interesting things to post about. Maybe I can even apply some of the things I am going to learn at my FS scenery design activities.

Object library mania

Since the release of EZ-Scenery I have seen quite a few discussion of people who asked if they could convert a Rwy12 library into a library for EZ-Scenery. Or I read about other people who started to decompile object libraries, so that they could create a EZ-Scenery library using the supplied library manager.

I find this a worrying development. EZ-Scenery is a wonderful tool, that allows someone who is not a hardcore scenery designer to place a few objects on his local airport very easily. But when those people start to decompile other libraries, it is very easy to create a complete chaos. Without knowing it, you could easily end up with the same object getting a lot of different GUIDs. You might even have the distribute different libraries with the same objects, because you chose the from library. This is certainly not the point of an object library. So therefore I want to make a few things about a library very clear:

  • An object library is an object library. There is not something like an EZ-Scenery library and a Rwy12 library. They are all the same. The only difference is that there are object libraries in the pre-Fs2004 format and in the Fs2004 (XML) format. But most modern programs (like Rwy12, EZ-Scenery, ObPlacer XML) use the Fs2004 format.
  • The purpose of an object library is that you have a collection of scenery objects, that can be used by any scenery. Just make sure that the user has all the required libraries installed.
  • Although it is very easy to extract a MDL object from a library and use them again in a different one, this is not something to encourage. Most scenery designers will see this as violating their copyright! So just ask your end users to install the same libraries you have installed and don’t start stealing stuff.

I know there are some differences in how the different placement tool use the object library. EZ-Scenery for example reads your scenery library and shows you all object libraries that are available. Rwy12 requires a special XML file that specifies which objects are available, while ObPlacer XML uses the XML source used when the library was created. But in the end they are all able to use the same object library. I really hope that we will see more general object libraries in the future, instead of libraries for Rwy12 or EZ-Scenery. It is time that people start to see they are all the same…

How to crash FS

I think every scenery designer will know how to do this. Just do something slightly wrong in your addon BGL file and FS will clearly tell you it does not like your new file. Especially when you start to tweak the ASM or SCASM source codes manual it is very easy to do something wrong.

But while working on MDL Tweaker II last evening I was able to make it crash even better. As you might know MDL Tweaker II does edit the MDL files directly, so I am writing the binary code to it directly. And while testing the feature to add conditions I by accident made a jump that jumped ahead 0 bytes. So it would jump to itself, and then jump again to itself, and then… Well I guess you can guess where this ends. FS clearly did not like this new MDL file (although BGLComp did not complain during compilation). Once I fired up Fs2004 the loading screen showed up for a little while and then I was back at my desktop, without any error or warning.

Now I understand a bit better why MS does not very actively encourage this kind of object tweaking. The chances that you make an error are quite large and their support will not appreciate problems like this as well. But that is no problem for me, I have already accepted that most of the crashes I get here are caused by myself. That is just the price you have to pay, when you try to extend the use of your scenery.

How about some presents from Santa?

Are you all anxiously waiting for you presents from Santa? I am not, as here in the Netherlands Santa is usually called Sinterklaas and already comes at the beginning of December (this is a long story, but actually both are the same. You can read more about it on Wikipedia). But this interesting story is not really the reason of my post.

I do have a week off between Christmas and New Year, so I plan to play for Santa a little bit myself. I want to finish some work I have been doing on my tools. There are some versions that are almost ready for release, expect a few small details. And I hope to iron these out in these few days. Tools that come to my mind here are the docking systems for ActiGate, the updated ObPlacer XML, the Library Creator 1.03 manual, CAT and of course MDL Tweaker II.

Somehow I have the feeling that those few days off might not be enough, but we’ll see…

OpenFlight

I want to start with a little warning. I am afraid this post is going to be rather long, so if you are already desperately in need of coffee now, you’d better get a cup before reading the rest of this.

So what is this OpenFlight you are talking about you might think? OpenFlight is not some sort of open source Flight Simulator or so. No it is a very common format to store your scenery in, for professional flight simulators. In this article I want to talk about the similarities between this format and the BGL format of Flight Simulator. And also about my attempts to make a converter between the two formats. Let’s get started.

The OpenFlight format has been create by MultiGen-Paradigm and is very common in real flightsimulators. At work we also make use of this format to store our visual databases (or scenery as most of you will know it). Rather soon after I started on my job, I had to learn more about the OpenFlight format, as I wanted to create a few tools to make our work easier. As I can almost dream the FS BGL/MDL format by now, it was rather logical that I compared this new format for me to what I knew. Slightly to my surprise a lot of things are almost the same. All materials are defined with the same properties (diffuse, ambient, specular and emissive colors for example) and the way to to store vertices and create triangles using them is also very similar.

The biggest difference for me was, that the OpenFlight format is a sort of a database. So each polygon has a few vertex nodes as children. And as a parent the polygon node can for example have a group, object or LOD node. This makes the structure of your object a lot clearer. A downside of this, is that the file format itself takes more diskspace. While learning about OpenFlight, I started to understand that MS has chosen a nice compact format to store the objects in.

There is another difference I should note. When I am talking about FS BGL/MDL files, I am talking about the object files mainly. The terrain has its own BGL format in FS, which has its own logic. In the OpenFlight format we usually create an entire database in one file (or a set of files that are linked together). So you have terrain, roads, 3D objects, airport all defined in the same file. As a result of this, the structure of your file is much more important, when trying to optimize your scenery. In FS scenery the MDL files usually contain only one object, so that makes their structure less important. All the placement is done with different (XML based) code and that has its own logic and structure again (I won’t start to talk about that as well here).

At work we were working on a database for Schiphol, the airport of Amsterdam. And because I had already created a FS scenery for this airport (as part of the NL2000 scenery team), the question soon came if we could not convert some of those 3D objects for use in the real flight simulator. One of the other NL2000 members had already created a converter the could create OpenFlight files from SCASM source code. But as we had created these objects with Fs2004 MDL files, they could not be decompiled to SCASM code. I had also started with the new version of my MDL Tweaker tool (which contains a nice MDL loader) at that moment, I decided that it would be cool if I could also add an OpenFlight export function to it.

In theory this conversion is not that hard. You have an object, that contains of a collection of vertices, materials, textures and triangles and you need to read the MDL format and write the geometry to the OpenFlight format. And the logic behind the conversion was indeed finished rather soon. But then the trouble started for me. I had to code this logic. The API provided with the OpenFlight format provides functions to create an OpenFlight file, but unfortunately for me these are in C++. And because I still need to find an easy way to create a nice GUI in C++, I am coding my MDL Tweaker project in Visual Basic.

So I first tried to use the functions of this API in Visual Basic. I won’t bother you with all technical details, but it proved to be more difficult then I had expected. Either the export function worked correctly, but crashed on big MDL files, or it worked on big files, but the properties in the OpenFlight format were not set correctly. So a few days ago, I decided that it would be easier if I wrote directly to the binary OpenFlight format from MDL Tweaker. A good discription of this binary format is luckily available.

So just this evening, I was able to finish that part of MDL Tweaker and the good news is that now everything works fine. So I can create OpenFlight files from my MDL scenery objects created with GMax. Now you also know the reason for the delay of a new alpha version of MDL Tweaker. The last few weeks I have mainly been busy to implement this new function (I could not accept that it did not work as I wanted).

After reading all this, you might think what can I do with all this? The answer for most of you is probably that the OpenFlight format is not of very much use to you. But if you want to play with it a little bit, I would advice you to give OpenSceneGraph a try. The OSGViewer that comes with OpenSceneGraph is able to load OpenFlight (FLT) files. So after I have released the next alpha version of MDL Tweaker, you can view your own objects in the OpenFlight format is you ever wish to do so :).

TerraVista

At work we use a tool called TerraVista to create the visual databases. This tool has some very interesting features, that are certainly part of my imaginary ultimate-scenery-design tool.

TerraVista for example allows you to import a lot of different data types. DEM altitude data, aerial or satelite photos, vector data for roads, etc. You can import them all, from a lot of different formats. If TerraVista knows the projection used in the data, it is able to combine them all for the project you are working on.

You can also determine rather flexibile how you want to use your data. Based on some attributes of your vector file, you can for example determine if it should be used to draw a road, a river or maybe a line of light poles. After you have defined these processing passes, the compiler takes care of the rest of the work, while it generates the visual database.

One of the things it can for example do during the generation is cut out different polygons. In Flight Simulator we are used to layer polygons if we have different ones on top of each other, but the Image Generator we use at work prefers to have only one polygon at a certain position. So TerraVista makes sure that the overlapping parts are cut away.

This is of course a very short discription of what the tool can do. It is so flexible that it sometimes even becomes hard to get something done, because you get lost in all the parameters that you can set.

But what about the ultimate-scenery-design tool? I would love to have a tool that I could provide with some altitude data, some photos, data for roads and that would then create the proper scenery files for me. Without me having to wonder about the LOD of the terrain and complicated things like that.

In that perfect imaginary world we would only need two kind of tools. One to design our 3D objects (GMax for example) and another tool that takes care of all the mesh and XML parts of the scenery. Maybe somebody would create that perfect tool in the future?

I know there are tools that can do part of this, but wouldn’t it be cool if we had all this in one tool? Especially for new designers it can be very frustrating to learn that they have to use yet another tool to get something done that sounds very simple at first.

As I design tools myself, I know how much work it would be to create a tool like this, that is powerfull on one hand and easy to use on the other hand. Maybe I can better keep dreaming…..

The projection hell

The different projection systems used in geographical data might be one of the aspects of MSFS scenery design that most designers know hardly anything about. But it surely a very important thing.

Almost each country has its own projection system and although most of these systems will give you the position in either degrees or meters, that does not mean that it will all fit together. So when using data to design your scenery it is a good idea to take a look at which projection system they use.

In the terrain SDK it is written that the MSFS mesh terrain uses the WGS84 projection system. But an interesting question remains which projection is used for the scenery objects (so the coordinates in meters you define in the MDL object file). As objects are in general small, the influence of the projection system will be a small, but it would still be interesting to know.

While working on my Dutch sceneries I first met all the trouble that different projections can give. We got some data that used the Dutch Rijksdriehoekscoordinaten system (the position is defined in meters from a church tower near the center of the country). But compared to the WGS84 projection system, the Dutch RD system also has an additional rotation of a few degrees. So when you think you can just use the position in meters, you end up with your scenery in the wrong location. In the end we luckily found a set of good formulas to do all the transformations to WGS84 and back.

Another example where you will meet different projection systems is when you use a background image in a program like Ground2k4 or SceneGenX. The mesh scenery and the XML style scenery are all defined in WGS84, but a lot of maps are in meters. So in that case you would have to reproject the image to get a perfect match. SBuilder is a tool that has added this reprojection functionality. If you do not take notice of this, you might be positioning your scenery at the wrong position, especially when you get further from the reference point.

For those that have become really enthousiastic about projection systems after reading this, take a look at the Proj4 library. With this library you can easily convert between almost any projection system. At work I have also used this library to build some conversion tools.