Jump to content
Slate Blackcurrant Watermelon Strawberry Orange Banana Apple Emerald Chocolate Marble
Slate Blackcurrant Watermelon Strawberry Orange Banana Apple Emerald Chocolate Marble
kancas55

Converting .bcs script files to .csv/plain text (Rune - Tanuki Soft Engine)

Recommended Posts

Hello, I'm attempting to translate some Tanuki Soft games, the game I'm testing right now is Shoukoujo. So far I've been able to extract all the files using Garbro and since the game reads files from folders inside the installation folder without needing to repackage them I've also been able to insert edited images.

The problem I'm having is with the .bcs script files, I took a look at the script files from Hatsukoi by Rune and I noticed they seem to share the same engine, both games use .bcs script files and if I open the files on a hex editor they seem to have the same header. Given how Hatsukoi has a fan translation where the .bcs files were converted to plain text saved as .csv, I believe doing the same for the script files of Tanuki Soft games would allow me to translate them.

My question is if there's anyone who can point me to where I could learn how to decrypt the .bcs files into plain text, maybe a way to reverse engineer the encryption given I have the original and decrypted files from the Hatsukoi patch, I would also like to know if there's anyone here from the Hatsukoi translation team, or anyone that knows what tools they could have used, that could maybe help me out.

Thanks.

Edited by kancas55

Share this post


Link to post
Share on other sites

Hey I'm new to this all decrypting business, but I have been trying to deal with .bcs files of RUNE games recently myself. The fact that there is a fan translation of one of their games gives me hope that my task is not insurmountable. But I have been hitting some bumps along the way. If you could link me to where I could download the original and the decrypted script of Hatsukoi I might be able to get a better understanding of whats going on there. Or at least narrow the possibilities.

Share this post


Link to post
Share on other sites

Here: https://mega.nz/#!4Lg3WCaQ!QDJhGXoo4qQIuwhxkR6XIbSqIxzS95028_yt79PXpkg

I also included some script files for the Tanuki Soft game, you can see how similar they are, like the 5 separate numbers in the header and the repeating pairs of data like EB FB during the beginning, the only difference is the Tanuki Soft script starts with the string TSV as opposed to BCS like in Hatsukoi, yet the files themselves still are .bcs.

Share this post


Link to post
Share on other sites
On 4/24/2019 at 10:28 AM, kancas55 said:

Here: https://mega.nz/#!4Lg3WCaQ!QDJhGXoo4qQIuwhxkR6XIbSqIxzS95028_yt79PXpkg

I also included some script files for the Tanuki Soft game, you can see how similar they are, like the 5 separate numbers in the header and the repeating pairs of data like EB FB during the beginning, the only difference is the Tanuki Soft script starts with the string TSV as opposed to BCS like in Hatsukoi, yet the files themselves still are .bcs.

Hi, im trying to translation another game from this artist that uses same engine. Can you re-upload those tools please?
 

Share this post


Link to post
Share on other sites

Sorry for necroing this thread, but I've been looking into .bcs files this past week and figured I'd post what I've managed to discover about them so far here in hope that someone much smarter than me can help figure out this format.

I've upload copies of .bcs files for Noshoujo, Hatsukoi, Shoukoujo, and Shoujo Ramune here: https://mega.nz/file/LF9GUCYD#UGUPyEzfDCmEGauyncCEiYwAi4t5OYhIViLfQc9pYAo

The Hatsukoi folder also has the translated CSVs available. Some of the translated files aren't incredibly helpful (like route data), since they're translations and a converted BCS wouldn't output the same file, but ones like V.csv should end up being the same file.

I've found that if it can't find datascn.tac then it tries to read in datascn.tac/_variable.csv and then datascn.tac/_variable.bcs. I was able to replicate that by removing the datascn.tac, confirming the game would no longer load, replacing it with a datascn.tac folder with the .bcs files inside, and confirming the game would once again load properly.

Digging into it a bit more with Ghidra, There is definitely some sort of tabular structure inside the .bcs files. It seems to be looking for certain column headers based on the .bcs file it's loading. For example:

_variable

  • %name%
  • %type%
  • %default%

_event

  • %no%
  • %label%
  • %cg%
  • %comment%
  • %type%

Like mentioned in the posts above, the files will either start the magic string TSV or BCS . Around the call to where it checks for that TSV magic string, strcmp(&v42, "TSV");, I see references to the vftable for TLibLZSS. Unfortunately I'm not nearly proficient enough at reverse engineering to work my way around the generated pseudocode to figure out what it's doing. I did notice that the first 24 bytes of the file appear to be the header. The first 4 being TSV\0, and then 5 more 4 byte integer fields, the last one likely being some sort of length property.

The assumption that the .bcs files are using some sort of LZSS compression seems to be along the right track, I found an archived posts from the Hatsukoi translation team that mentions a variant of LZSS being used:
https://warosu.org/jp/thread/3247959#p3259250
https://warosu.org/jp/thread/3247959#p3260079

Since the threads are so old by this point, a lot of the links and images are dead-ends, including what appears to be the holy grail in terms of a link to the source code of a BCS to CSV conversion program:
https://warosu.org/jp/thread/3310174#p3325324

The source code for the LZSS variant, spiruneg, seems to be alive and well on GitHub. https://github.com/gocha/spiruneg
Though my attempts to throw some BCS data into LSZZ decoder in spiruneg never resulted in producing anything useful. The data it spits outs appears to be mostly 0x00 and 0xFF, and I couldn't see much else of note.

 

Edited by IlIIIIlllIlIl

Share this post


Link to post
Share on other sites
On 1/22/2021 at 1:03 PM, IlIIIIlllIlIl said:

Sorry for necroing this thread, but I've been looking into .bcs files this past week and figured I'd post what I've managed to discover about them so far here in hope that someone much smarter than me can help figure out this format.

*snip*

 

Coincidentally I've also been chipping away at this for a while, and I've already gotten the scripts out of the Tanuki Soft games(and Rune, but I'm less interested in those). I was in the process of trying to write a tac archive generator but since you found out that it's possible to just make a folder named datascn.tac and put the files there, that makes it a lot easier. The problem I ran into now is that the Tanuki Soft games don't appear to parse csv in the same way as the old Rune games. I'm testing with Bishoujo and Shoujo Ramune, but the game either cannot find certain csvs(and will complain that it can't find the scenario), or it will crash. I believe the crash is from some parsing failure.

I've uploaded the csvs from datascn.tac in Bishoujo here: https://mega.nz/folder/cfwk1DiJ#2d8KFJ3x0qw7KetrAm711g in case others want to experiment as to why the game refuses to take in the csvs. I've found that _music.csv and _chapter.csv actually do work without errors, but _project.csv doesn't even though it's a small file and has near identical format to the _PROJECT.CSV from Musume Shimai.

From reverse engineering the binary and testing with 01common.csv, it appears to run into an invalid vector error eventually but I haven't found much else. The other option if I can't get the csvs to work is to write a bcs generator but that could take a while. I'll keep working on it in the meantime, I plan to throw it onto github once I'm all done.

Share this post


Link to post
Share on other sites
On 1/25/2021 at 4:17 AM, NEL1391 said:

Coincidentally I've also been chipping away at this for a while, and I've already gotten the scripts out of the Tanuki Soft games(and Rune, but I'm less interested in those). I was in the process of trying to write a tac archive generator but since you found out that it's possible to just make a folder named datascn.tac and put the files there, that makes it a lot easier. The problem I ran into now is that the Tanuki Soft games don't appear to parse csv in the same way as the old Rune games. I'm testing with Bishoujo and Shoujo Ramune, but the game either cannot find certain csvs(and will complain that it can't find the scenario), or it will crash. I believe the crash is from some parsing failure.

I've uploaded the csvs from datascn.tac in Bishoujo here: https://mega.nz/folder/cfwk1DiJ#2d8KFJ3x0qw7KetrAm711g in case others want to experiment as to why the game refuses to take in the csvs. I've found that _music.csv and _chapter.csv actually do work without errors, but _project.csv doesn't even though it's a small file and has near identical format to the _PROJECT.CSV from Musume Shimai.

From reverse engineering the binary and testing with 01common.csv, it appears to run into an invalid vector error eventually but I haven't found much else. The other option if I can't get the csvs to work is to write a bcs generator but that could take a while. I'll keep working on it in the meantime, I plan to throw it onto github once I'm all done.

Hmm, it looks like there's actually two issues going on here.

The first I discovered that when trying to load the standard .bcs files in the datascn.tac folder and only including _variable.csv there would be a bunch of pop-ups complaining that it didn't find variables that were being referenced. Looking at the _variable.csv file, it's clear it was trying to define those variables, so I stepped through it with a debugger and it was actually treating the csv file like it was one giant line. So I passed the csv file through unix2dos to change the line endings from LF to CRLF and it stopped complaining and played fine.

Once I started incorporating the rest of the CSV files the only ones that caused it to crash were: _define.csv, _stand.csv, 01common.csv, 02tamaki.csv, 03kyoka.csv, 04haruna.csv, and macro.csv.

Back into the disassembler, I saw the same std::_Xout_of_range("invalid vector<T> subscript"); exception that you're getting. Specifically, it looks like it's erroring because it's not handling quotation marks correctly. In the function that raises the Xout_of_range exception, it runs through that function once first and parses it as you would expect for a csv file. So just below the Xout_of_range exception, you'll see a ton of comparisons to 0x22 (hexadecimal quote). I haven't stepped through that with the debugger yet, so I don't know where exactly in that function it's failing to parse.

I think the easiest csv file to use to look into it is _stand.csv, since the only quotation marks are on line 29 in what appears to be a comment. Trying to start it with _stand.csv included will crash the program, but removing the two quotes from that line and starting it again will let it run fine. It's also worth noting that I was able to replace each quotation mark with two quotation marks to escape them, as expected from csv.

That said, you can apply to other files as well, such as with 01common.csv:

GDRUOdu.png

Share this post


Link to post
Share on other sites
2 hours ago, IlIIIIlllIlIl said:

Hmm, it looks like there's actually two issues going on here.

The first I discovered that when trying to load the standard .bcs files in the datascn.tac folder and only including _variable.csv there would be a bunch of pop-ups complaining that it didn't find variables that were being referenced. Looking at the _variable.csv file, it's clear it was trying to define those variables, so I stepped through it with a debugger and it was actually treating the csv file like it was one giant line. So I passed the csv file through unix2dos to change the line endings from LF to CRLF and it stopped complaining and played fine.

Once I started incorporating the rest of the CSV files the only ones that caused it to crash were: _define.csv, _stand.csv, 01common.csv, 02tamaki.csv, 03kyoka.csv, 04haruna.csv, and macro.csv.

*snip*

 

Interesting, I missed the obvious line ending one there. That fixed one of them, alright. If double quoting can work, then that could probably be used entirely instead of figuring out how exactly they parse csvs. Adding double quotes does work in _stand.csv, but trying the same in 01common.csv is still giving me a crash. Did you just add double quotes wherever there were quotes in 01common.csv?

Share this post


Link to post
Share on other sites
9 hours ago, NEL1391 said:

Interesting, I missed the obvious line ending one there. That fixed one of them, alright. If double quoting can work, then that could probably be used entirely instead of figuring out how exactly they parse csvs. Adding double quotes does work in _stand.csv, but trying the same in 01common.csv is still giving me a crash. Did you just add double quotes wherever there were quotes in 01common.csv?

Most lines seem to be fine within the file, I didn't have to do anything to the $fset lines at the start:

10200 ,"$fset %f_tamaki%,0",,,,,,,,,,,,,,,,珠希進行度=0

The lines I had to change were ones like:

57500 ,,"<!se_24_knc_people00><shake id="0" target="sc" time="700,700" x="12,300,0" y="60,240,0">",,,,,,,,,,,,,,,どんっ!

I got the game to be fine with it after I changed that to:

57500 ,,"<!se_24_knc_people00><shake id=""0"" target=""sc"" time=""700,700"" x=""12,300,0"" y=""60,240,0"">",,,,,,,,,,,,,,,どんっ!

I didn't play through it to verify that the shake effect was still functioning, but I imagine it'd still work.

The other thing I noticed was that any Japanese text in the file would be garbled when loading through the csv file, but that was easily fixed by saving the files as Shift-JIS instead of UTF-8.

Share this post


Link to post
Share on other sites
4 hours ago, IlIIIIlllIlIl said:

Most lines seem to be fine within the file, I didn't have to do anything to the $fset lines at the start:

10200 ,"$fset %f_tamaki%,0",,,,,,,,,,,,,,,,珠希進行度=0

The lines I had to change were ones like:

57500 ,,"<!se_24_knc_people00><shake id="0" target="sc" time="700,700" x="12,300,0" y="60,240,0">",,,,,,,,,,,,,,,どんっ!

I got the game to be fine with it after I changed that to:

57500 ,,"<!se_24_knc_people00><shake id=""0"" target=""sc"" time=""700,700"" x=""12,300,0"" y=""60,240,0"">",,,,,,,,,,,,,,,どんっ!

I didn't play through it to verify that the shake effect was still functioning, but I imagine it'd still work.

The other thing I noticed was that any Japanese text in the file would be garbled when loading through the csv file, but that was easily fixed by saving the files as Shift-JIS instead of UTF-8.

I was missing the final conversion to shift-jis, I was converting individual strings to shift-jis but was writing to the file in utf-8. Nice catch, I think that's all the outstanding issues then. I'll clean up a bit and upload to a repo.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...