scenProc filter bug

I have fixed the scenProc filter bug that I wrote about a few weeks ago. Objects without an attribute were sometimes included in the results for double or integer attribute types. That behaviour has been fixed now.

While fixing it one new issue appeared though. The filter validation now doesn’t throw an error anymore when you try to compare a string to an integer. But I didn’t want to hold back the bug fix until I have been able to fix this issue. So the validation is slightly less good for now.

GUID selection context menu

In scenProc the code completion will help you to select the right GUIDs for your vegetation or buidlings. But if you are not sure what the name of the class is or if you want to add multiple guids for random placement, that’s not always the easiest way.

So I have now added a context menu to these GUID attributes and selecting it will give you a GUI like shown below where you can easily select the GUID you want from the list. Just check the checkbox and the GUID will be added to your script.


Find and replace

I have added find and replace functionality to the scenProc GUI now. That should help you to find a certain text in your configuration file and also to make replacements more quickly. You’ll find those new functionalities in an extra toolbar that is shown right below the normal toolbar.


Filtering features

Better ways to filter out features in scenProc that are overlapping with others or that are close together have been requested for a while already and I have now implemented a new step called FilterFeatures that adds these possibilities.

With this new step you can for example remove all point features of trees that are close to a round or remove buildings that have bounding boxes that are overlapping with other buildings. Please check the manual for the details and examples of how to use this new step.

Some of this filtering could be done before by using the AddAttributeIfInside step, for example to add an extra attribute when a feature was inside another feature and then filter them out based on this attribute. But the new step will make this kind of filtering a lot more efficient.

Understanding extrusion bridges

Lately I have been working with XML extrusion bridges for scenery and while doing so I noticed some weird artifacts. Even though I out the end points under the ground, they sometimes still float above it. And I also noticed that two bridges don’t always connect even though you specify the same position and altitude for the connecting point.

So I decided to dive a little deeper and see if I can understand what is going on. And after examining a lot of bridges I have come to the conclusion that the scenery engine adds a weird elevation offset to all bridges. And this offset is equal to the average height of all points of the bridge. So the taller your bridge is, the more the bridge is raised and thus also the more your end points will float.

The good news is now that I understand the logic behind the offset it is easy to compensate for it, to make sure you get the bridge you expect. But I still don’t understand why it was implemented like this. I mean when you type an altitude in the XML file that’s what you expect to get, right?

Removed some steps

I last nights build I have deprecated some steps that were marked as to be depcated for a while already. This includes the ExportSHP step. Since alternative steps are available for a while already this should not really give problems anymore. The reason to deprecate them now is that I found out that ExportSHP is not working correctly with the attributes in all cases and I don’t want to spend more time on fixing that. 

ImportAGN and autogen distortion

It’s two years ago already that we figured out that Annotator does not display the orientation of autogen as FS does. There is a distortion in the angle of the objects. So since this was figured out scenProc applies a correction to the orientation to make sure everything aligns correctly with the world.

But what if you use the ImportAGN step to load autogen into scenProc? If you load the autogen as autogen objects scenProc just keeps them in memory (so that you can add new autogen objects) and saves them again if you use ExportAGN. So in that case the distortion correction is no problem, it will just be preserved.

But if you use the option in ImportAGN to create features from the autogen then you will have an issue. When creating features scenProc will convert the autogen from the autogen coordinates to geodetic coordinates again. Until now the correction for the autogen distortion was not included in those calculations, which would give an error. So I have now updated scenProc to correctly “uncorrect” the distortion correction.

If you are creating features from autogen not made with scenProc (so autogen that doesn’t have the correction), you can add the  NOINVERSEDISTORTION option to the ImportAGN step to not apply the “uncorrection”.

Improved validation

With the new filter syntax I introduced about a month ago, I did also see an increase in error reports due to issues with the filter syntax. Most of the time these errors where caused by user input (sorry to say this), for example because in the filter it was tested if a string is bigger than an integer or because the filter only listed a variable name but without a condition.

I have now improved the validation, so that these errors will also be reported while you type your script. To be able to do this I had to implement a “quick attribute scan” feature that will check your source data for the types of the attributes. Since this involves opening your geo data files I decided to schedule this check a bit less frequent than the normal validation, else the performance would probably be hurt too much. So it could take a few seconds before all errors related to the filter appear.

Hopefully these changes help you to write the correct filter syntax more easily. And hopefully it also reduces the amount of errors being reported.

New scenProc features

From tomorrow the scenProc 2.0 development release will have some new features included. Here is an overview of the main changes.

Polygon autogen library objects

The main change is that autogen library objects are now created from polygon instead of point features. This is in line with how Annotator creates these objects as well. The benefit of this is that you can now also create autogen library objects from building footprint data more easily. With the new CreateAGNLibObject step you can create the autogen library objects from polygons. The old way to create them from point is still available with the CreateAGNLibObj step that will be deprecated in the near future.

PointToPolygon step

Since library objects are made from polygons now, there are situations where you need to turn point features into polygon features. For this a new step has been added PointToPolygon. This step will create polygons with the length and width that you specify.

The existing ExtrudeLine step has been renamed to LineToPolygon step as well, so that the steps to convert between feature types have more logical names. The existing steps to create points from lines will be renamed to LineToPoint in the future as well to make everything more logical.

Updated tooltips

A third change is that I have changed the way that the help tooltips about the steps are generated. Before the information for the help text, the argument types and the sample line were spread in different places of my code. They are now in one place and that should make sure they stay in sync more (some of the help text was a bit outdated). So this new change should make the tooltips even more clear and accurate. But since I made a lot of changes now to the code, please let me know if some mistakes have slipped in.


Is it easy to waste memory?

mugI guess you already know the answer to the question posted in the title of this post. I’m afraid the answer was yes. This week two users (thanks to both of you) reported to me that scenProc is quite hungry for memory, especially when processing big files. Even 16 GB of RAM would easily be filled in certain cases.

So I started to check where all this memory went and the reason was quite logical. ModelConverterX and scenProc share a lot of code, they use the same libraries for many of the processing functions.  But the data structures designed initially for ModelConverterX were not always optimal for scenProc as well. In ModelConverterX for each vertex information like the position, normal, texture coordinate and bone weight are stored. scenProc doesn’t need all this information, only the position is enough. But it was using the same class to represent the vertices. So that gave a huge waste of memory.

I have now optimized this and also changed a few other things how scenProc stores the lines and polygons. And the good news is that this results in a much better memory usage. A reduction of 50% to 70% is possible for most scripts. So I’m sure you’ll notice the difference. And processing times also slightly improved in most cases because the information is stored more efficiently.

So grab the development release tomorrow and have a look yourself. Oh, and please don’t fill up the RAM that you freed with even more data. Because I’m sure some of you will go to the limit again 😉