Data extraction thread

[ShiroChikara 0]

Hello my friend, i cannot go on the page of Tlwiki, is down the server¿ can you help witch that¿

[loam 13]

I heard that some were having trouble extracting .dat files that come with Aokana and AokanaEXTRA1, despite it apparently being based on another known format from what I've heard.

Here's a tool that allows you to both unpack .dat files to folders and folders back to .dat files that will work with the game if you decide to modify its contents. It supports both the PC and Switch formats. Early versions of this tool were used to make the Aokana +18 patch for the switch, which consisted in simply unpacking the PC +18 files and pack them into the Switch's version format.

Binary: http://www.mediafire.com/file/0zgkf38g3dswoy6/AokanaDatTool.zip/file

Source code (C#, .NET Core): http://www.mediafire.com/file/51oelx9qq91gh6s/AokanaDatToolSrc.zip/file

Edited November 10, 2020 by loam

[iGSfYMkbtk 1]

On 11/8/2020 at 6:49 AM, loam said:

I heard that some were having trouble extracting .dat files that come with Aokana and AokanaEXTRA1, despite it apparently being based on another known format from what I've heard.

Here's a tool that allows you to both unpack .dat files to folders and folders back to .dat files that will work with the game if you decide to modify its contents. It supports both the PC and Switch formats. Early versions of this tool were used to make the Aokana +18 patch for the switch, which consisted in simply unpacking the PC +18 files and pack them into the Switch's version format.

Binary: http://www.mediafire.com/file/6ssm4q8m2kb1qsi/AokanaDatTool.zip/file

Source code (C#, .NET Core): http://www.mediafire.com/file/wq6cn4j6zfrr74t/AokanaDatToolSrc.zip/file

Hello!
I tested the program and found a problem about "repack" part of this tool:
The output *.dat file seems not be accepted at all or loaded correctly by Windows-Steam version of both Aokana and Extra 1!
The CGs (or even maybe sounds and scripts too?) inside repacked part of the game shows up as gray screen,
Even without modifying anything and then just straight up pack them back immediately!

Proven by unpack all of the 12 unpackable *.dat files in "AokanaEXTRA1_Data" from Windows-Steam Extra 1,
then pack them back with this tool, replace the original *.dats,
then the game won't even start the startup sequence after launch the game...(whole screen just went black.)
Commandline used: "AokanaDatTool.exe" --pack --input "unpacked_root_folder" --output "repacked.dat"

I also tested to unpack the repacked *.dat from this tool back to a different folder,
then compares with the unpacked file hashes from the original *.dat, and nothings different...
Then i tested another known working script from GitHub:
https://github.com/hktkqj/Aokana-Extractor
Use that to unpack the original games *.dat to another folder again,
compares with the result from this tool but again... All of the files hashes are the same...
So the "unpack" part of this tool is ok.

(BTW, the unpacking speed of this tool is significantly faster than that script from GitHub mentioned above!
Great job man! Sorry for my bad english...)

Edited November 9, 2020 by iGSfYMkbtk

[loam 13]

14 hours ago, iGSfYMkbtk said:

Hello!
I tested the program and found a problem about "repack" part of this tool:
The output *.dat file seems not be accepted at all or loaded correctly by Windows-Steam version of both Aokana and Extra 1!
The CGs (or even maybe sounds and scripts too?) inside repacked part of the game shows up as gray screen,
Even without modifying anything and then just straight up pack them back immediately!

Proven by unpack all of the 12 unpackable *.dat files in "AokanaEXTRA1_Data" from Windows-Steam Extra 1,
then pack them back with this tool, replace the original *.dats,
then the game won't even start the startup sequence after launch the game...(whole screen just went black.)
Commandline used: "AokanaDatTool.exe" --pack --input "unpacked_root_folder" --output "repacked.dat"

I also tested to unpack the repacked *.dat from this tool back to a different folder,
then compares with the unpacked file hashes from the original *.dat, and nothings different...
Then i tested another known working script from GitHub:
https://github.com/hktkqj/Aokana-Extractor
Use that to unpack the original games *.dat to another folder again,
compares with the result from this tool but again... All of the files hashes are the same...
So the "unpack" part of this tool is ok.

(BTW, the unpacking speed of this tool is significantly faster than that script from GitHub mentioned above!
Great job man! Sorry for my bad english...)

Hey, thanks for reporting this. The bug was caused because the paths were being written with a trailing \ into the .dat file when packing them, because my code is removing the base path without considering that and just removing what you give it in the --input. If you use --input "unpacked_root_folder\" with the broken version, you will notice it will work properly. I used this tool like that while making the Switch +18 patch. This is clearly not the intended behavior, so I've fixed it:

Updated download links:

Binary: http://www.mediafire.com/file/0zgkf38g3dswoy6/AokanaDatTool.zip/file

Source: http://www.mediafire.com/file/51oelx9qq91gh6s/AokanaDatToolSrc.zip/file

Thanks for letting me know!

PS: I didn't knew a python script existed out there already, but it's clear there wasn't an attempt to understand what the code actually did. I tried to make the code as clear as possible with meaningful variable/function names so I hope somebody finds that useful.

Edited November 10, 2020 by loam

[GTX1080Ti 0]

Hi everyone. I'm trying to extract the opening/ending "image sequence" or raw file from the games, Yosuga no Sora, Haruka na Sora, Natsu no Ame.

It seems the opening is stored in the opening.mei file (the file name kinda gives it away). By using GARbro I am able to see a stream of images inside, however its not able to extract it in png format. Is there any tool which can deal with this? A quick google led me to believe this is Entis/ERI format, but literally no pointers to how I can get something out of it.

[Kelebek1 106]

3 hours ago, GTX1080Ti said:

Hi everyone. I'm trying to extract the opening/ending "image sequence" or raw file from the games, Yosuga no Sora, Haruka na Sora, Natsu no Ame.

It seems the opening is stored in the opening.mei file (the file name kinda gives it away). By using GARbro I am able to see a stream of images inside, however its not able to extract it in png format. Is there any tool which can deal with this? A quick google led me to believe this is Entis/ERI format, but literally no pointers to how I can get something out of it.

How big is the file? I downloaded the game trial but it doesn't have any opening.mei file in it, seems like it's not included with the trial. Is the file too big to upload?

[GTX1080Ti 0]

Yosuga no Sora - 66843KB

Haruka na Sora - 96542KB

Natsu no Ame - 80255KB

They're not really very big.

Edited November 13, 2020 by GTX1080Ti

[Flare 0]

Hi guys! I would like to know if I can ask for help about extracting data of a otome game in this topic. There's archives that I've never seen, I really need the help of someone that know about programming. If i'm in the wrong topic, please let me know and I'll delete this. Thanks for reading

[GuyInDogSuit 0]

Does anyone know how to extract images from the English version of Riddle Joker? I end up with unusable files like "ev102a.pimg" and GARbro says they're archives, but I've no idea how to extract them.

[Zakamutt 1359]

On 12/7/2020 at 1:50 AM, Flare said:

Hi guys! I would like to know if I can ask for help about extracting data of a otome game in this topic. There's archives that I've never seen, I really need the help of someone that know about programming. If i'm in the wrong topic, please let me know and I'll delete this. Thanks for reading

https://dontasktoask.com

[Flare 0]

Sorry if that was a dumb question. I'm just new in forums so I don't really know how to ask for help in them.

The game that i'm translating is "Hakuoki Kyoto Winds", a visual novel otome in Steam. I saw the archives and I think the ones to translate the game are ".pac" (STORY_EN00000.pac , GAME_EN00000.pac to GAME_EN00004.pac) and ".cpk" files (SYSTEM_EN.cpk , GAME_EN.cpk , STORY_EN.cpk).

Do you know any tools to extract data from this files and edit them? If you could recommend me ones and give me a little help to learn how to use it I would really appreciate it! Thanks for reading

[GuyInDogSuit 0]

I know this is a long shot, seeing as how morkt hasn't been active for months (since March 30, apparently), but IxSHE Tell (English) just came out on Steam and I can't seem to extract any image files from it successfully. I mean, they DO extract, but then they're not open-able. The archives are in the root.pfs format (Artemis Engine?) and while GARbro recognizes them as archives and allows me to look into them and extract them, the images themselves come out as if they're still.... encrypted. Every other file format works fine, though. For some odd reason, I can only successfully view or extract backgrounds. Audio (ogg) and such plays just fine, too.

Also.....
Does anyone know how to extract images from the English version of Riddle Joker? I end up with unusable files like "ev102a.pimg" and GARbro says they're archives, but I've no idea how to extract them.

[Qooappboy 0]

On 11/9/2020 at 7:20 PM, loam said:

Hey, thanks for reporting this. The bug was caused because the paths were being written with a trailing \ into the .dat file when packing them, because my code is removing the base path without considering that and just removing what you give it in the --input. If you use --input "unpacked_root_folder\" with the broken version, you will notice it will work properly. I used this tool like that while making the Switch +18 patch. This is clearly not the intended behavior, so I've fixed it:

Updated download links:

Binary: http://www.mediafire.com/file/0zgkf38g3dswoy6/AokanaDatTool.zip/file

Source: http://www.mediafire.com/file/51oelx9qq91gh6s/AokanaDatToolSrc.zip/file

Thanks for letting me know!

PS: I didn't knew a python script existed out there already, but it's clear there wasn't an attempt to understand what the code actually did. I tried to make the code as clear as possible with meaningful variable/function names so I hope somebody finds that useful.

 When ever I try to use this program it always Crashes on me when I put the .Dat file on top of the (AokanaDatToolextractor). Any advice?

[GuyInDogSuit 0]

On 12/26/2020 at 1:28 AM, GuyInDogSuit said:

I know this is a long shot, seeing as how morkt hasn't been active for months (since March 30, apparently), but IxSHE Tell (English) just came out on Steam and I can't seem to extract any image files from it successfully. I mean, they DO extract, but then they're not open-able. The archives are in the root.pfs format (Artemis Engine?) and while GARbro recognizes them as archives and allows me to look into them and extract them, the images themselves come out as if they're still.... encrypted. Every other file format works fine, though. For some odd reason, I can only successfully view or extract backgrounds. Audio (ogg) and such plays just fine, too.

Also.....
Does anyone know how to extract images from the English version of Riddle Joker? I end up with unusable files like "ev102a.pimg" and GARbro says they're archives, but I've no idea how to extract them.

 

Bump?

[Freezey 0]

Does anyone know how to use asmodean's tools?, specifically exnllpk for the .lpk format. I'm able to extract the script file but can't figure out how to use the gameinit.sob for the key that is needed to extract the others. Link for exnllpk

 

Edited January 17 by Freezey
Problem solved

[rinnow 0]

On 12/26/2020 at 6:28 PM, GuyInDogSuit said:

I know this is a long shot, seeing as how morkt hasn't been active for months (since March 30, apparently), but IxSHE Tell (English) just came out on Steam and I can't seem to extract any image files from it successfully. I mean, they DO extract, but then they're not open-able. The archives are in the root.pfs format (Artemis Engine?) and while GARbro recognizes them as archives and allows me to look into them and extract them, the images themselves come out as if they're still.... encrypted. Every other file format works fine, though. For some odd reason, I can only successfully view or extract backgrounds. Audio (ogg) and such plays just fine, too.

Also.....
Does anyone know how to extract images from the English version of Riddle Joker? I end up with unusable files like "ev102a.pimg" and GARbro says they're archives, but I've no idea how to extract them.

I guess you can use this

https：//github.com/sst311212/exixshepng

[Sigma57f 4]

Hi everyone! So, um, I'm trying to translate Remember 11 to my language (spanish), but I'm having trouble with the script files.

These are packed in an AFS file which I managed to extract with a tool. The problem is that they're in .BIP format and I can't find any way to edit them. I tried searching for several days by now, but the closest I've gotten ended up in me not knowing how to convert them into .BIP again (and I'm not even sure it would've worked, either way). 

So, if anyone knows a tool or some way to edit the script, I'd really really appreciate it! 

[GuyInDogSuit 0]

On 1/22/2021 at 9:34 PM, rinnow said:

I guess you can use this

https：//github.com/sst311212/exixshepng

 

Thanks! Is it supposed to only be able to convert one at a time, though? There's almost no instructions on how to use it otherwise.

[WinkelHime 0]

Hello, need some help with Eiyuu Senki/ES Gold modding. 

I'm using arc_conv and it's working fine, except for one file - AUEvent/Face.dat. I can unpack it, like other ES .dat files, but after repacking it back (even without adding/changing anything inside) game shows errors on startup and crashing on start of campaign.

Back in 2016, when we worked on ES translation port form PS3/Vita, I remember having same issue, but our teammate wolfmoren somehow managed to properly repack it (and we added PS3/Vita characters to the game). Alas, 4 years is a long time, Hongfire died, I lost contact with wolfmoren and our private messages about ES modding got deleted on forum we used.

Without ability to modify Face.dat, properly adding new characters is impossible, which severely limiting any additional modding I would like to do.

I appreciate any help/advice, thanks.

[Tester 4]

I have created a new GUI tool SLGSystemDataTool for quite obscure for hackers or coders engine — SLG System. Legendary series Sengoku Hime and Sangoku Hime are on this engine (probably all Gesen 18 games are on it, but it'll need some testing). Unlike of my previous tools, this one does not support scripts (I did manage to decompile Sengoku Hime 1's scripts, but they are rather tricky, and there was some complications, so I did not release it).

What can it do?

- Extract all types of archives (szs, and SFP (SPD+SPL).
- Repack all archives.
- Convert all types of images (tig, tic, TIM or even alb) to standart formats.
- Reconvert all types of images from standart formats.
- Convert and reconvert VOI audio (it has ogg and some technical data).
- Reconverted alb are half (or less than a half) of original size. It was reached by abusing technology used in this exotic image format.

It also has a help module, which has info about working with video mdm and mda (the tool is not required) besides other formats.

It would be appreciated if the top post would be updated accordingly.

Supported formats: szs, SFP (SPD+SPL), tig, tic, TIM, alb, VOI.
Engine: SLG System.

Edited February 17 by Tester

[rinnow 0]

On 2/14/2021 at 6:46 PM, GuyInDogSuit said:

 

Thanks! Is it supposed to only be able to convert one at a time, though? There's almost no instructions on how to use it otherwise.

https：//gist.github.com/sst311212/3ac36d712457251ff8bdb76ca2a7d9c1

Use this .bat.

Also I'm not sure but did you try krkrextract for riddle joker?

[NewbTransla 0]

Hi! 

Anybody knows what type of engine does ANIM.TEAM.MM games use?

Is there a way to extract the .dat scripts? Hopefully somebody can help me.