I guess you already know the answer to the question posted in the title of this post. I’m afraid the answer was yes. This week two users (thanks to both of you) reported to me that scenProc is quite hungry for memory, especially when processing big files. Even 16 GB of RAM would easily be filled in certain cases.

So I started to check where all this memory went and the reason was quite logical. ModelConverterX and scenProc share a lot of code, they use the same libraries for many of the processing functions.  But the data structures designed initially for ModelConverterX were not always optimal for scenProc as well. In ModelConverterX for each vertex information like the position, normal, texture coordinate and bone weight are stored. scenProc doesn’t need all this information, only the position is enough. But it was using the same class to represent the vertices. So that gave a huge waste of memory.

I have now optimized this and also changed a few other things how scenProc stores the lines and polygons. And the good news is that this results in a much better memory usage. A reduction of 50% to 70% is possible for most scripts. So I’m sure you’ll notice the difference. And processing times also slightly improved in most cases because the information is stored more efficiently.

So grab the development release tomorrow and have a look yourself. Oh, and please don’t fill up the RAM that you freed with even more data. Because I’m sure some of you will go to the limit again 😉

The next development release of scenProc will contain the new filter syntax. Please have a good look at the updated manual, as it shows how the new syntax works and contains updated samples of all steps. Here are just the highlights of the changes:

• Besides And conditions, you can now also do Or or Not conditions.
• Different attribute types are now supported, before every attribute was seen as a string, but now you can also have integers or doubles.
• You can do math within your filters, so for example sum two attributes or apply a math function on an attribute.
• scenProc will automatically try to update the old filter syntax and in 95% of the cases that should work fine.
• One of the biggest changes to be aware of when writing the filter is that if you compare to a text, you need to put quotes around it

I hope you all get used quickly to the new syntax, because it is a lot more powerful. And if there are any questions just post them in the forum.

Just a early warning, the scenProc update with the new filter syntax is almost ready for release. I have just started to update the manual for all the changes, so hopefully in a few days I can put this feature in the development release.

Why this warning? This update will change how the filter syntax works, which might impact your existing scripts. There is logic in the tool to translate from the old to the new syntax and in 98% of the cases that should work. But if you want to be really sure, you better keep using scenProc 1.1 for your production work.

If however you want to use the new features (Or selections in the filter, the ability to do math with attributes in the filter, the ability to use Math functions in your filter) you should have a look at the new filter syntax once it’s released.

And now back to writing the manual…..

The scenProc development release has now been updated to version 2.0. One of the first changes I have pushed into the development release is an update of the GDAL libraries to a more recent version. So if you grab the development release 2.0 for the first time, please do a full download and not an update download.

For the rest there should only be minor changes compared to the 1.1 version. The big changes to the filter syntax and to the way library objects are generated are still in my development branches and will make it to the development release once they have been tested more.

I have just released the stable version of scenProc 1.1. Compared to the old stable release 1.0 the changes made recently have been included. These are not really big changes, but since scenProc 2.0 will have some bigger changes coming up soon, I have decided to release a new stable release that people can fall back to.

You can find all the download links at the scenProc page. And in a few days I’ll update the development release to version 2.0 and start to roll out the new features I have been working on recently.

I have now modified the ImportOGR step of scenProc. For the projection you can now specify a value of AUTODETECT to tell scenProc to try to detect the projection used by itself. This works for most vector data formats that GDAL/OGR supports, as long as the projection data is specified correctly in the vector data. For example for shapefiles there is often a PRJ file that specifies the projection of the data.

If the automatic detection fails you can still specify it manually like it was done before. And the NOREPROJ value to specify the projection is already WGS84 geodetic coordinates is also still supported of course.

Until now the scenProc ImportGDAL step required that the imagery you are reading was in WGS84 projection already. I have now changed this and other projections are also supported. If the imagery is not in WGS84 it will be reprojected by scenProc internally. This should make it even easier to run the feature detection feature on the imagery you have, as many sources don’t provide the imagery in WGS84 projection. So it should save quite some pre-processing of your data.

scenProc will try to automatically detect the projection that is used in your imagery. In most cases that works fine, if this doesn’t work out you can manually provide the projection as an additional argument to the ImportGDAL step.

There is one little catch, the internal reprojection is done in memory. So if you are already short on memory for your scenProc script, it might be better to still reproject your imagery manually. But from the tests I have done until now this shouldn’t be needed too often.