Sins of a Solar Empire : Real-Time Strategy. Unrivalled Scale.
© 2003-2016 Ironclad Games Corporation Vancouver, BC. All rights reserved.
© 2006-2016 Stardock Entertainment

{utility release} GUI for convertdata to automate the conversion of files in folders between BIN & TXT

harpos TXT BINner *v1.19.12* as at 15/3/2010

By on May 23, 2009 6:19:26 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

harpo99999

Join Date 02/2008
+405

I have started coding on another little program for automated conversion of files between txt & bin formats using the actual convertdata*.exe's and the correct convertdata for the files based on the pathname eg for entrenchment mods/reference files the path should contain "entrenchment" ditto for the next expansions, and for vanilla sins it SHOULD NOT contain any of the expansions names also it will work on all drives in the computer BUT does need the sins installation in the location pointed to by the registy.

it will also be able to make TXTed reference files as easily as converting an entire mod to/from bin

to make your own reference set you first COPY the program files sins folder to another location in the computer like for example 'my documents', then delete FROM THE COPY the appdata,font & sdplugins folders and the dll and exe files in the root of the copy then run the txt binner doing the entrenchment folder first then any other expansions you have and finally the vanilla sins and for each folder note which files have FAILED conversion

a more detailed overview of the program

psudo code outline

get the conversion type

get the location to convert

determine which convertdata to use based on the location (all three expansions will be supported)

scan the location to create a list of all files

for each file in the list use the convertdata to convert each file if it is NOT the type wanted, then test to see if it was converted correctly, if not list it in the failed conversion list, if it was  then list it in the success list

end psudo code outline

note additional minor feature if you have a location you want to use as a sins reference files location and want it to be accessably with a single button then create a sinsref.txt in the same folder as the txt binner.exe and have as the first line of the file as the path to your sins reference location eg 'f:\sins reference'  note the LACK of trailing backslash the program does not want it

the new versions of the mod updater and TXT BIN GUI use the correct version of the convertdata to do the actual convesion to text/bin eg updateing mods from entrenchment 1.01 to 1.03 the mod updater converts to text with convertdata_entrenchment1.01.exe and updates the entities, then converts back to bin with convertdata_entrenchment 1.03.exe.

I suggest not using the old version files the current is 1.1

the latest version is included in the utility pack that an be downloaded from my mod updater thread

 

harpo

my new txt bin converter GUI

Locked Post 238 Replies +2
Search this post
Subscription Options


Reason for Karma (Optional)
Successfully updated karma reason!
May 24, 2009 12:07:24 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

nice :

Reason for Karma (Optional)
Successfully updated karma reason!
May 24, 2009 8:29:10 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

a bit more info the actual conversion process is fairly slow and takes about 10-20 minuitesto procss a copy of entrenchment between bin & txt. I am currently trying to improve the performance of the program

harpo

 

Reason for Karma (Optional)
Successfully updated karma reason!
May 25, 2009 6:16:34 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

further updates

had some "fun" sorting out the txt to bin conversion as the converted to txt files have 3 nontext characters at the start of the file before the TXT, and the binary files start with BIN. the conversion from txt is working and now the conversion from txt is working

conversion times for a example of the 1.02 entrenchment full file set is about 3-5 minuites for txt to bin and 3-10 minuites for bin to txt with most of the time spent in the convertdata*.exe's.

also the program uses the correct convertdata for the sins version and picks the expansion based on the file path. ie if entrenchment is in the path then the entrenchment convertdata is used, if entrenchment is not in the path then the vanilla sins convertdata.exe is used.

also the program will be able to support all three expansions in spite of the lack of knowledge about the names of the expansions and their folders, I will have to release updated versions after the beta for the expansions starts( ie when I know the folder names and convertdata names

I am going to increase the size of the gui from fitting in a 640*480 screen to an 800*600 screen to make the conversion results windows easier to follow

harpo

 

Reason for Karma (Optional)
Successfully updated karma reason!
May 25, 2009 7:51:36 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

latest info

on my development computer(amd x2 5200+ 4gb ddr800 gfeorce7600 xp32bit sp3) it took 5 minutes to convert all of entrenchment 1.02 to txt and a run with no changes takes less than 10 seconds and a convert of all files takes approximatly 5 minutes to convert to BIN

the current outline of the program is

1 get the location to convert and the type of conversion,

2 determine the best converter to use based on the folder names in the path( at present wether or not entrenchment is on the path),

3 then work through the types of files that can be converted in the folder AND subfolders of the folder creating a list of files that might need to be converted,

4 then work through the list testing each file to see if it is in the WANTED format ie txt or bin,

5 if the file is NOT in the wanted format then run the convertdata*.exe converting the file,

6 if the file IS in the wanted format then skip the converter and add it to the list of skipped files,

7 after the convertdata run test to see if the file is NOW in the wanted format, if it is add to the list of susessfull conversions, if it is not then add it to the list of failed convserions,

8 work on the next file

9 after testing & converting all the files calculate how long it took to do the conversions and inform the user

 

the program is almost at beta status as it will convert the entrenchment files and the sins files either by skipping or converting based onthe header of the file an I can not see any major problems either in my code OR the code interfacing with the convertdata exe's

harpo

 

 

Reason for Karma (Optional)
Successfully updated karma reason!
May 25, 2009 8:15:12 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

further timing info

the convseion of the sins 1.16 from bin to txt took the development machine 10.9 minutes with three files failing the conversion

CAPITALSHIP_PHASEBATTLESHIP.entity
CAPITALSHIP_PHASECARRIER.entity
CAPITALSHIP_PHASECOLONY.entity

and the convertdata used was the convertdata_originalsins.exe

and I did copy the failed files list and pasted it into this post.

harpo

 

Reason for Karma (Optional)
Successfully updated karma reason!
May 26, 2009 7:21:51 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

the sins txt binner program is about ready for the beta phase, if ambody would like to help me beta test this please pm me

harpo

 

Reason for Karma (Optional)
Successfully updated karma reason!
May 27, 2009 4:37:29 AM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

harpo:

Do those three match the list I gave you of ships in the 1.16 folder you gave me, and I found a couple BINs in? I forget where I posted that.

Reason for Karma (Optional)
Successfully updated karma reason!
May 27, 2009 6:54:29 AM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

yes they are the same ships, but from vanilla sins, but I am cetain that the files I emailed you are txt, I just do not remember how I had sucessfully converted them, and the thread was ask danman page 10 reply 250, andthe source of the problem is that the convertdata_originalsins.exe would not convert them. I do not know why

harpo

 

Reason for Karma (Optional)
Successfully updated karma reason!
May 27, 2009 11:08:29 AM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

I don't know modding, but do want to look at the .entity files to see info I can use for the Sins wiki. Can somebody help a total mod n00b - is this GUI something I could use to do this? And what is this GUI - a file downloaded from the Library?

Ultimately all I need is one simple text conversion of all the .entity files. I'm not interested in changing them. Currently I only have vanilla Sins but may get Entrenchment sooner or later.

Thanks if you can help. And thanks for the utility, harpo... it sounds very powerful!

Mike

Reason for Karma (Optional)
Successfully updated karma reason!
May 27, 2009 11:28:01 AM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

Harpo99999: Just doublecheck that they are text files, not BINs. Just a thought.

NoSpamMan: No, I'm afraid this is not useful for that. What sort of stats do you want to give to the wiki? You may do well to obtain a parser that grabs the relevant info from each file; though it must be warned this would lack all the Lore surrounding them and even the actual names (Without a lot of fiddling).

Reason for Karma (Optional)
Successfully updated karma reason!
May 27, 2009 1:12:48 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

Hi Reiver,

Yes, I know what you mean. For some tables like  http://sinsofasolarempire.wikia.com/wiki/Planet_Types#Planet_Bonuses (hmm... why can't I replace the link with text?), I had to combine info from 3 files, including English.str. A bit of work, but not too much.

I can turn all the .entity files into text wih a parser? Cool - can you point me to one? Hopefully, one that will convert them all in one batch job.

I did that work approx. a year ago from v. 1.02, using some .entity text conversions that somebody had placed in the library at that time. But I can't remember where and don't know if that conversion gets updated with each update (which I doubt). Whatever works... all I need is a text conversion of them.

BTW somebody apparently just changed the old wiki (http://www.soasewiki.com) to being hosted by wikia. Those mistakes are not mine, like where is says the Planet Bonuses are from v. 1.2 (should say 1.02). They also apparently dropped a lot of my good research, wah.

Thanks if you can help!

Reason for Karma (Optional)
Successfully updated karma reason!
May 27, 2009 6:30:31 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

reiverta the program DOES check if the files have been correctly converted and if not then are listed as failed to convert

psudocode outline

determine which expansion is being worked onthen select correct convertdata*.exe

get filename

check file to see what type it is. if file is wanted type then skip and add to skip count & list

convertdata*.exe used to convert file to wanted type

check file to see what type it is, if file is wanted type then add to sucess count & list

if file NOT wanted type then add to fail count & list

repeat for all files in subfolders

end psudocode.

nospamman gui is a Graphical User Interface or wrapping program to make a commandline program easier for newer users to use, and this one is aimed more at modders that need txted reference files before the official reference files are released, and can convert a COPY of the sins files to the TXT format which is one of the tests that I have been using, as an example back in reply 5 I listed 3 entity files that failed to convert from BIN to TXT in a test run on a complete copy of sins 1.16, so far I have not had any entrenchment files fail to convert. the three files that failed in the 1.16 run are the ones that usually fail convertdata conversion in 1.16.

harpo

 

 

Reason for Karma (Optional)
Successfully updated karma reason!
May 27, 2009 7:57:14 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

Thanks, Harpo. And apologies for asking about something else in your thread - I had hoped it would be a quick answer. Anyway I peeked in the ForgeTools3 download and I see all the .entity files for 1.16 in text. So I'm good.

Thanks for your cool GUI!

Reason for Karma (Optional)
Successfully updated karma reason!
May 27, 2009 8:08:19 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

nospamman the forge tools are for 1.15 NOT 1.16, I hope that forgetools will be released for the next patch as well,also ask the questions where you think they will get answered. I am one of the modders that have uploaded txted un-official reference files for both sins & entrenchment. the un-officials are on both my mediafire and filefront pages

harpo

 

Reason for Karma (Optional)
Successfully updated karma reason!
May 27, 2009 8:16:17 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

So is this program an extension of your manifest maker? Could the two be merged?

Reason for Karma (Optional)
Successfully updated karma reason!
May 27, 2009 8:27:24 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

rieverta the program is similar but different, and the two could be merged but at the cost of MORE buttons to confuse users, increasing the size of the program, increasing the complexity and updating difficulty.

I always try to Keep It Simple for Slow people, but I prefer Keep It Clean and Simple for Slow people.

the easier it is to use te more people that can use it effectively

harpo

Reason for Karma (Optional)
Successfully updated karma reason!
May 29, 2009 1:55:59 AM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

Fair enough.

Reason for Karma (Optional)
Successfully updated karma reason!
May 29, 2009 7:28:58 AM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

I have just added a text file report of the results of the conversion run in three txt files eg sucessTXT.txt, failTXT.txt, skippedTXT.txt for the bin to text conversion and sucessBIN.txt, failBIN.txt, and skippedBIN.txt for the text to bin conversions and they are placed in the mod/reference folder that was processed.

I thought that a report file of what wa and was not done was a good idea so added it. if there are any other ideas I will consider them and if I think that they might benefit a modder then I will add them

harpo

 

Reason for Karma (Optional)
Successfully updated karma reason!
May 30, 2009 3:13:34 AM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

I have just found & fixed a problem with the textfile that points to the reference files containing an invalid path crashing the program. now it the reference file path is invalid it is ignored

harpo

 

Reason for Karma (Optional)
Successfully updated karma reason!
May 30, 2009 1:26:31 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

kudos to you, Harpo... thank you for all your work. i'm chomping at the bit waiting for this!

Reason for Karma (Optional)
Successfully updated karma reason!
June 3, 2009 4:07:13 AM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

just another update.

there has been no modifications/bugfixes on the program in the last 4 days so I am thinking of releasing it in three days unless any of my beta testers find any more problems or ideas

so PLEASE try to find problems and ideas that might improve it

harpo

 

Reason for Karma (Optional)
Successfully updated karma reason!
June 4, 2009 2:31:33 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

havent found any since the last update it works great

 

Reason for Karma (Optional)
Successfully updated karma reason!
June 4, 2009 5:45:45 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

I have decided to upload the final version as v1.0 on my download pages mediafire, game fanatics and filefront

use & enjoy

harpo

edit gamingvault has closed

Reason for Karma (Optional)
Successfully updated karma reason!
June 10, 2009 6:31:04 PM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

media fire has deleted all of my files so I WILL NOT be uploading anything to them again

harpo

Reason for Karma (Optional)
Successfully updated karma reason!
June 14, 2009 11:20:34 AM from Sins of a Solar Empire Forums Sins of a Solar Empire Forums

I was curious, does this work for dds files and textures? the program doesn't seem to want to convert those.

Reason for Karma (Optional)
Successfully updated karma reason!
Stardock Forums v1.0.0.0    #101114  walnut1   Server Load Time: 00:00:00.0000250   Page Render Time: