Data extraction thread

[ShiroChikara 0]

Hello my friend, i cannot go on the page of Tlwiki, is down the server¿ can you help witch that¿

[loam 12]

I heard that some were having trouble extracting .dat files that come with Aokana and AokanaEXTRA1, despite it apparently being based on another known format from what I've heard.

Here's a tool that allows you to both unpack .dat files to folders and folders back to .dat files that will work with the game if you decide to modify its contents. It supports both the PC and Switch formats. Early versions of this tool were used to make the Aokana +18 patch for the switch, which consisted in simply unpacking the PC +18 files and pack them into the Switch's version format.

Binary: http://www.mediafire.com/file/0zgkf38g3dswoy6/AokanaDatTool.zip/file

Source code (C#, .NET Core): http://www.mediafire.com/file/51oelx9qq91gh6s/AokanaDatToolSrc.zip/file

Edited November 10, 2020 by loam

[iGSfYMkbtk 1]

On 11/8/2020 at 6:49 AM, loam said:

I heard that some were having trouble extracting .dat files that come with Aokana and AokanaEXTRA1, despite it apparently being based on another known format from what I've heard.

Here's a tool that allows you to both unpack .dat files to folders and folders back to .dat files that will work with the game if you decide to modify its contents. It supports both the PC and Switch formats. Early versions of this tool were used to make the Aokana +18 patch for the switch, which consisted in simply unpacking the PC +18 files and pack them into the Switch's version format.

Binary: http://www.mediafire.com/file/6ssm4q8m2kb1qsi/AokanaDatTool.zip/file

Source code (C#, .NET Core): http://www.mediafire.com/file/wq6cn4j6zfrr74t/AokanaDatToolSrc.zip/file

Hello!
I tested the program and found a problem about "repack" part of this tool:
The output *.dat file seems not be accepted at all or loaded correctly by Windows-Steam version of both Aokana and Extra 1!
The CGs (or even maybe sounds and scripts too?) inside repacked part of the game shows up as gray screen,
Even without modifying anything and then just straight up pack them back immediately!

Proven by unpack all of the 12 unpackable *.dat files in "AokanaEXTRA1_Data" from Windows-Steam Extra 1,
then pack them back with this tool, replace the original *.dats,
then the game won't even start the startup sequence after launch the game...(whole screen just went black.)
Commandline used: "AokanaDatTool.exe" --pack --input "unpacked_root_folder" --output "repacked.dat"

I also tested to unpack the repacked *.dat from this tool back to a different folder,
then compares with the unpacked file hashes from the original *.dat, and nothings different...
Then i tested another known working script from GitHub:
https://github.com/hktkqj/Aokana-Extractor
Use that to unpack the original games *.dat to another folder again,
compares with the result from this tool but again... All of the files hashes are the same...
So the "unpack" part of this tool is ok.

(BTW, the unpacking speed of this tool is significantly faster than that script from GitHub mentioned above!
Great job man! Sorry for my bad english...)

Edited November 9, 2020 by iGSfYMkbtk

[loam 12]

14 hours ago, iGSfYMkbtk said:

Hello!
I tested the program and found a problem about "repack" part of this tool:
The output *.dat file seems not be accepted at all or loaded correctly by Windows-Steam version of both Aokana and Extra 1!
The CGs (or even maybe sounds and scripts too?) inside repacked part of the game shows up as gray screen,
Even without modifying anything and then just straight up pack them back immediately!

Proven by unpack all of the 12 unpackable *.dat files in "AokanaEXTRA1_Data" from Windows-Steam Extra 1,
then pack them back with this tool, replace the original *.dats,
then the game won't even start the startup sequence after launch the game...(whole screen just went black.)
Commandline used: "AokanaDatTool.exe" --pack --input "unpacked_root_folder" --output "repacked.dat"

I also tested to unpack the repacked *.dat from this tool back to a different folder,
then compares with the unpacked file hashes from the original *.dat, and nothings different...
Then i tested another known working script from GitHub:
https://github.com/hktkqj/Aokana-Extractor
Use that to unpack the original games *.dat to another folder again,
compares with the result from this tool but again... All of the files hashes are the same...
So the "unpack" part of this tool is ok.

(BTW, the unpacking speed of this tool is significantly faster than that script from GitHub mentioned above!
Great job man! Sorry for my bad english...)

Hey, thanks for reporting this. The bug was caused because the paths were being written with a trailing \ into the .dat file when packing them, because my code is removing the base path without considering that and just removing what you give it in the --input. If you use --input "unpacked_root_folder\" with the broken version, you will notice it will work properly. I used this tool like that while making the Switch +18 patch. This is clearly not the intended behavior, so I've fixed it:

Updated download links:

Binary: http://www.mediafire.com/file/0zgkf38g3dswoy6/AokanaDatTool.zip/file

Source: http://www.mediafire.com/file/51oelx9qq91gh6s/AokanaDatToolSrc.zip/file

Thanks for letting me know!

PS: I didn't knew a python script existed out there already, but it's clear there wasn't an attempt to understand what the code actually did. I tried to make the code as clear as possible with meaningful variable/function names so I hope somebody finds that useful.

Edited November 10, 2020 by loam

[GTX1080Ti 0]

Hi everyone. I'm trying to extract the opening/ending "image sequence" or raw file from the games, Yosuga no Sora, Haruka na Sora, Natsu no Ame.

It seems the opening is stored in the opening.mei file (the file name kinda gives it away). By using GARbro I am able to see a stream of images inside, however its not able to extract it in png format. Is there any tool which can deal with this? A quick google led me to believe this is Entis/ERI format, but literally no pointers to how I can get something out of it.

[Kelebek1 106]

3 hours ago, GTX1080Ti said:

Hi everyone. I'm trying to extract the opening/ending "image sequence" or raw file from the games, Yosuga no Sora, Haruka na Sora, Natsu no Ame.

It seems the opening is stored in the opening.mei file (the file name kinda gives it away). By using GARbro I am able to see a stream of images inside, however its not able to extract it in png format. Is there any tool which can deal with this? A quick google led me to believe this is Entis/ERI format, but literally no pointers to how I can get something out of it.

How big is the file? I downloaded the game trial but it doesn't have any opening.mei file in it, seems like it's not included with the trial. Is the file too big to upload?

[GTX1080Ti 0]

Yosuga no Sora - 66843KB

Haruka na Sora - 96542KB

Natsu no Ame - 80255KB

They're not really very big.

Edited November 13, 2020 by GTX1080Ti

[Flare 0]

Hi guys! I would like to know if I can ask for help about extracting data of a otome game in this topic. There's archives that I've never seen, I really need the help of someone that know about programming. If i'm in the wrong topic, please let me know and I'll delete this. Thanks for reading

[GuyInDogSuit 0]

Does anyone know how to extract images from the English version of Riddle Joker? I end up with unusable files like "ev102a.pimg" and GARbro says they're archives, but I've no idea how to extract them.

[Zakamutt 1,348]

On 12/7/2020 at 1:50 AM, Flare said:

Hi guys! I would like to know if I can ask for help about extracting data of a otome game in this topic. There's archives that I've never seen, I really need the help of someone that know about programming. If i'm in the wrong topic, please let me know and I'll delete this. Thanks for reading

https://dontasktoask.com

[Flare 0]

Sorry if that was a dumb question. I'm just new in forums so I don't really know how to ask for help in them.

The game that i'm translating is "Hakuoki Kyoto Winds", a visual novel otome in Steam. I saw the archives and I think the ones to translate the game are ".pac" (STORY_EN00000.pac , GAME_EN00000.pac to GAME_EN00004.pac) and ".cpk" files (SYSTEM_EN.cpk , GAME_EN.cpk , STORY_EN.cpk).

Do you know any tools to extract data from this files and edit them? If you could recommend me ones and give me a little help to learn how to use it I would really appreciate it! Thanks for reading

[GuyInDogSuit 0]

I know this is a long shot, seeing as how morkt hasn't been active for months (since March 30, apparently), but IxSHE Tell (English) just came out on Steam and I can't seem to extract any image files from it successfully. I mean, they DO extract, but then they're not open-able. The archives are in the root.pfs format (Artemis Engine?) and while GARbro recognizes them as archives and allows me to look into them and extract them, the images themselves come out as if they're still.... encrypted. Every other file format works fine, though. For some odd reason, I can only successfully view or extract backgrounds. Audio (ogg) and such plays just fine, too.

Also.....
Does anyone know how to extract images from the English version of Riddle Joker? I end up with unusable files like "ev102a.pimg" and GARbro says they're archives, but I've no idea how to extract them.

[Qooappboy 0]

On 11/9/2020 at 7:20 PM, loam said:

Hey, thanks for reporting this. The bug was caused because the paths were being written with a trailing \ into the .dat file when packing them, because my code is removing the base path without considering that and just removing what you give it in the --input. If you use --input "unpacked_root_folder\" with the broken version, you will notice it will work properly. I used this tool like that while making the Switch +18 patch. This is clearly not the intended behavior, so I've fixed it:

Updated download links:

Binary: http://www.mediafire.com/file/0zgkf38g3dswoy6/AokanaDatTool.zip/file

Source: http://www.mediafire.com/file/51oelx9qq91gh6s/AokanaDatToolSrc.zip/file

Thanks for letting me know!

PS: I didn't knew a python script existed out there already, but it's clear there wasn't an attempt to understand what the code actually did. I tried to make the code as clear as possible with meaningful variable/function names so I hope somebody finds that useful.

 When ever I try to use this program it always Crashes on me when I put the .Dat file on top of the (AokanaDatToolextractor). Any advice?

[GuyInDogSuit 0]

On 12/26/2020 at 1:28 AM, GuyInDogSuit said:

I know this is a long shot, seeing as how morkt hasn't been active for months (since March 30, apparently), but IxSHE Tell (English) just came out on Steam and I can't seem to extract any image files from it successfully. I mean, they DO extract, but then they're not open-able. The archives are in the root.pfs format (Artemis Engine?) and while GARbro recognizes them as archives and allows me to look into them and extract them, the images themselves come out as if they're still.... encrypted. Every other file format works fine, though. For some odd reason, I can only successfully view or extract backgrounds. Audio (ogg) and such plays just fine, too.

Also.....
Does anyone know how to extract images from the English version of Riddle Joker? I end up with unusable files like "ev102a.pimg" and GARbro says they're archives, but I've no idea how to extract them.

 

Bump?

[Freezey 0]

Does anyone know how to use asmodean's tools?, specifically exnllpk for the .lpk format. I'm able to extract the script file but can't figure out how to use the gameinit.sob for the key that is needed to extract the others. Link for exnllpk

 

Edited 12 hours ago by Freezey
Problem solved