Short term work - What should happen to managed content?

All right folks. I don't know if I want to continue development of openBVE in the long run, but I think I could spend some time on making a short term maintenance release. What concerns me most is the now partially broken managed content system. While we have uploaded backups of the database to our servers, the database was not updated since May 8th. Which means it's becoming more and more worthless unless we decide what to do with it. I have the tools to continue running the system if I wanted to. But since the whole system wasn't received well by developers anyway, it's a valid question if we should just scrap the system entirely. I see the following choices:

1) Dismantle managed content. Meaning the main menu would not show the Get Add-Ons menu any longer. The Start New Game menu would only show Add-Ons if there are any and otherwise hide the tab and just go directly into Browse Manually. Effectively this would mean that already installed Add-Ons would continue to work (so that existing users aren't pissed off), but downloading and installing them through the game would not be possible any more because we wouldn't maintain the Add-On directory server.

2) The other extreme. We continue to maintain the managed content directory server with daily updates as was the case during the last half year. But this would only make sense if more developers start using managed content. I would be inclined to remove the licensing requirements and just check packages for integrity.

3) A partial managed content system. Add-Ons can be installed automatically, but there wouldn't be a server to store the list of Add-Ons. This means that developers would publish their Add-Ons on personal homepages, forums, file-sharing sites etc., and that users will have to find those sites as usual. But after downloading an Add-On, it wouldn't be in .zip, .7z, .rar or .exe, but in a special package format for automatic installation. For those who know Omsi, it would be somewhat like an .ams file. After downloading the package, you could either double-click it or use an Install From File... button in openBVE to automatically install the Add-On. Besides from the raw files, the package would also contain the additional metadata needed for categorization such as country, city, description and links back to the author's site. The package would be created with a special Add-On Manager tool that ensures correct packaging and allows to fill in the metadata. If desired, developers could also provide an update capability. There would be a Check For Updates button in openBVE which when clicked would contact all sites that support automatic updates and could show available updates to users like Firefox does with Extensions.

If you have an opinion, post it. I will take it into consideration.

Option 1 is easiest to implement. As you say, the whole system wasn't really taking off, there was probably no single reason but rather several obstacles preventing it from working successfully. All that needs to be addressed before reintroducing it in any form.

It is no use at the moment, leaving it in when it doesn't work is pointless.

Any system, for it to be successful, must be EASY for content providers who aren't necessarily computer geniuses to implement SIMPLY. If more of us had understood how to package content there might have been more enthusiasm to provide it. I didn't provide anything for managed content because I just couldn't understand it and would not risk messing around altering my files and delivery systems which are working perfectly well. And I haven't a clue who or what Omsi is!

Some developers also though there were licensing incompatibilities with the system too.

So initially, remove it please. Then IF a SIMPLE to use partially-managed content system can be devised I, for one, would be happy to provide content for it so long as it is within my limited technical grasp and provided it was within the scope of the CC licensing on all celtictrainsim.com products.

Well, the licensing requirements could be loosened, or we could have a system of repos (and preferably, there should be some sort of official list of good repos)

odakyufan wrote:All right folks. I don't know if I want to continue development of openBVE in the long run, but I think I could spend some time on making a short term maintenance release. What concerns me most is the now partially broken managed content system. While we have uploaded backups of the database to our servers, the database was not updated since May 8th. Which means it's becoming more and more worthless unless we decide what to do with it. I have the tools to continue running the system if I wanted to. But since the whole system wasn't received well by developers anyway, it's a valid question if we should just scrap the system entirely. I see the following choices:

1) Dismantle managed content. Meaning the main menu would not show the Get Add-Ons menu any longer. The Start New Game menu would only show Add-Ons if there are any and otherwise hide the tab and just go directly into Browse Manually. Effectively this would mean that already installed Add-Ons would continue to work (so that existing users aren't pissed off), but downloading and installing them through the game would not be possible any more because we wouldn't maintain the Add-On directory server.

Option 1 - With the content mirrored to a known long-term repository, so content isn't "lost". I assume you will also be making a maintenance release of Chisanai ?

In time, content managment should probably be more like 'package managers' for Linux, Debian has an open source deb format. Alternatives
are source code management systems like Subversion.

I'd also strongly suggest establishing an 'official' source code repository. There seems to be an 'unofficial' Subversion/Git repository for OpenBVE's code by virtue of the fact that someone made a debian package at some point..

Also would it be possible to examine if it's possible to release two other items ?
1) Was a technology preview of a true 3D environment Route viewer , done by michelle as a 'quick' test .
2) A text editor (by michelle)- which to date has been the only 'free' text editor I've found that's been able to do proper import from different vendor code pages !! (I've got an archive copy of this somewhere, including the source.)

some sort of official list

I think that is a non-runner unless the Official OpenBVE site returns operationally. You can't have an 'official list' if the office is closed! The closest thing we have in practical terms is BVEWiki, which is all kinds of BVE, including many works for the older sim versions, some dating back quite a long way.

odakyufan wrote:

3) A partial managed content system. Add-Ons can be installed automatically, but there wouldn't be a server to store the list of Add-Ons. This means that developers would publish their Add-Ons on personal homepages, forums, file-sharing sites etc., and that users will have to find those sites as usual. But after downloading an Add-On, it wouldn't be in .zip, .7z, .rar or .exe, but in a special package format for automatic installation. For those who know Omsi, it would be somewhat like an .ams file. After downloading the package, you could either double-click it or use an Install From File... button in openBVE to automatically install the Add-On. Besides from the raw files, the package would also contain the additional metadata needed for categorization such as country, city, description and links back to the author's site. The package would be created with a special Add-On Manager tool that ensures correct packaging and allows to fill in the metadata. If desired, developers could also provide an update capability. There would be a Check For Updates button in openBVE which when clicked would contact all sites that support automatic updates and could show available updates to users like Firefox does with Extensions.

Assuming that you are able to define a spec for the metadata, this could just be an additional file added to a defined archive format.
This metadata-file could then be looked for specfically in the archive. There's no need to create a new archive format, all it would need
is for the OpenBVE code to detect a specific named file in an archive directory. Are you familiar with tools like gzip and tar?

What metadata about route/train/object packages is needed ?

Are you familiar with tools like gzip and tar?

No.

What part of "SIMPLE" do you not understand?

graymac wrote:

Are you familiar with tools like gzip and tar?

No.

What part of "SIMPLE" do you not understand?

Gzip and tar are open and simple. And if not, there's always 7zip.

I use 7zip compression for files intending downloading.
If any managed or semi managed system relied on the downloadable content being packaged in the well understood conventional form of
Railway - Object/Route/Sound then would that suffice? I know some material isn't done like that, but it should be.
If, as an example, the url to download the Ballyfeckin route files is http://www.celtictrainsim.com/ctsdownloads/trainitems/object.7z then why can't the content manager latch on to that? That link results in a zipfile download, which unzips to give you a folder called "Ballyfeckinv2". That being a route, it only has to be put in the users route folder.
Surely that is the basis of a workable system?

graymac wrote:

Are you familiar with tools like gzip and tar?

No.
What part of "SIMPLE" do you not understand?

The idea I had was that " PlatformTwo" was able to read a manifest DIRECTLY out of the relevant 7Z file (Like Java can when downloading applets in JAR files.) or externally.

The comment about gzip and tar was trying to determine if Odakyufan was familiar with the source code for those programs, Under NO circumstances should a content developer have to know about compression tools. 'Packaging' should make that part opaque to both
developers and users..

Drag0nflamez wrote:

graymac wrote:

Are you familiar with tools like gzip and tar?

No.

What part of "SIMPLE" do you not understand?

Gzip and tar are open and simple. And if not, there's always 7zip.

Gzip and tar are NOT simple to non geeks. Most of the time when I've made archives, I've used UI interfaces..

The most important thing is to keep it VERY simple. I several developers (including myself) were reluctant to put their add-ons on the managed content system because it was a complicated endeavor to do so.
There should be nothing regarding the use of metadata, tars, gzips, etc.
At the most, developers should just have to drag and drop folders to an online server in order to get it there content online.


With regards to the hosting of content, a simplification of the managed content system to facilitate more add-ons being uploaded would result in an increase in the amount of online space required to store said add-ons. It is important to remember that hosting space will need to be provided (and backed-up).

alex_farlie wrote:
The idea I had was that " PlatformTwo" was able to read a manifest DIRECTLY out of the relevant 7Z file (Like Java can when downloading applets in JAR files.) or externally.

Speaking of which : - http://blog.logiclabz.com/c/unzip-files-in-net-c-using-sharpziplib-open-source-library.aspx
Extracting an optional metadata file becomes rather easy to do...

That said, to keep things SIMPLE i find myself in agreement with Greymac here.

You see what I mean here? It is plain that what the programming brains (what some call "geeks") and the bulk of the route-makers have different definitions of "simple"
This is exactly WHY the 'managed content' addition didn't fly when it was introduced.
I think Alex can see that. Just because some of us have fought tooth-and-nail to be able to tackle a .csv file it DOESN'T mean we can understand all that C# stuff. I can manage C# on a piano, the other sort is a foreign language.

I'll go for something between 2 and 3.

There will be an get add-on menu, which downloads add-on packages from the links in the database, however, openBVE also has a Open File function to access locally stored pacakages too.

Also, there should be a new developer tool to assist in the creation of the descriptor file to include in the packages. This was something tht was missing in the first version of managed content.

But since that's only the unzipping part and you only need to write a small simple file then zip it using a simple tool, shouldn't be a issue.

Drag0nflamez wrote:But since that's only the unzipping part and you only need to write a small simple file then zip it using a simple tool, shouldn't be a issue.

In the OMSI example provided, the "Importable OMSI Addon" .ams files are simply .zip files, renamed to the .ams extension. There is no need for a tool to pack things up, as it's as simple as two clicks in Windows Explorer. The descriptor file generator are more essential though.

@Odakyufan - I don't think anybody outside Germany knows Omsi. @ Everyone out of Germany round here (that would be everyone except Odakyufan and me); Omsi is a bus simulator. However, I do not think that's the right way. Why don't we do it as Graymac said it? The dev uploads a zip/7z/... archive with the common, known structure on their homepage, into a folder they specify in a commonly known list (be it hosted on opebve.org, be it hosted here, be it hosted by Odakyfan). This way, virtually nothing changes for the dev. The rest is done by the server hosting the list; a program on the server checks all linked folders on the dev's homepages (e.g. timbuktubve.freesites.to/packages) for new or changed zip files and lists them in the "Get Addons" tab, letting the user select all updates and new addons they want and download them.

That would mean neither does the dev need to learn anything new, so it meets the requirements Graymac specified, nor does this disrupt the old-style distribution, as anyone can choose to instead download the zip file and install as earlier. Optionally one might think about giving the dev an option to add a "package.txt" and "package.png" into the zip file, which would be shown in the "Get Addons" tab but not extracted into the OpenBVE content folder; simply as a mean of describing the package, linking their homepage or specifying the license.

I think that option 1 is the easiest to implement and especially easier to maintain.

The maintenance of the database is inevitably more expensive with the increase in the number of entries.

On the other hand, according to the requests for help that I receive daily by Italian newbies, the first installation of the program is the main problem, not the addons.

Personally,I still believe that, at the very least, include some form of offline managed content like those .ams files.

But why? I don't see the sense of country and company listing, this is, in my eyes, an open invitation to chaos. We need a feature to download and install addons through the program; but I think, this should happen inside the "LegacyContent" folder, not through that strange new system. Graymac exactly wrote what we need: Anything, as long as the dev does not need to learn some new complicated system noone knows how well it works.

I'm shifting anything between 16-25Gb of downloads from celtictrainsim.com monthly. That's a lot of folks downloading the Ballyfeckin routes and trains all over the world.
And the number of querys I've had concerning difficulty with downloads is about three or four. And I think that was because those users hadn't downloaded the BRSigs component. Which is now included in the Objects as well as being obtainable off the site as a "fix".

If the program managers could have accepted a URL like the example above and below, here:
http://www.celtictrainsim.com/ctsdownloads/trainitems/item.7z
Then there would have been some more to augment the other very lonely items available from within the program.

i've had over 6,000 350 downloads, which even my co-partner didnt believe until i gave him permission to access my downloads.

Since them i haven't been able to count due to shifting to another file host site.

graymac wrote:If the program managers could have accepted a URL like the example above and below, here:
http://www.celtictrainsim.com/ctsdownloads/trainitems/item.7z
Then there would have been some more to augment the other very lonely items available from within the program.

That's exactly what I propose! I do not have the slightest clue why Michelle did it the complicated way.