Data extraction thread

[otchi]

Does anyone know how to unpack data from games amde iwth Visual Novel Maker engine? In the particular game I'm looking at, the resources are actually "open", but the fle format is encrypted. The file would say "PNG" but it cannot be opened. And when I looked at it with a Hex editor, it's detected as a mysterious file format .

 

On 2/5/2019 at 9:44 AM, sena_kaito said:

Using this tool,
http://dnrv2.cyriaca.net/makuta/projects/hello/hate2.exe

How to use,
Open CMD on hate2 folder
Then type:
hate2 "input file" "output folder" (without quotes)
hate2 X:\img.dat X:\img (example)
And voila!

Nb: maybe you need turn off your antivirus, in some case, they treat as fake detection.

Good luck!

Sorry for quoting an old post, but I saw this link and when I tried it, it's dead.

Do you know an alternative link for it?

Edited October 30, 2021 by otchi

[Triggerhappy]

Is this thread still useful? Honestly i have a lot of time on my hands and a fairly decent grasp of the Japanese language so i want to start working on some translations in my spare time to both help others play these games and improve my skill at the same time, my problem has always been this part of it though - actually pulling out the files and stuff that you need.

For example the visual novel i am looking at doing only has three files, live.dll, a save and a huge exe file. And the opening post basically doesn't go in to this at all beyond Crass which is seemingly quite hard to find now especially to get it working. Is there anything more up to date you guys can recommend to read up on?

Edited November 14, 2021 by Triggerhappy

[rinnow]

22 hours ago, Triggerhappy said:

Is this thread still useful? Honestly i have a lot of time on my hands and a fairly decent grasp of the Japanese language so i want to start working on some translations in my spare time to both help others play these games and improve my skill at the same time, my problem has always been this part of it though - actually pulling out the files and stuff that you need.

For example the visual novel i am looking at doing only has three files, live.dll, a save and a huge exe file. And the opening post basically doesn't go in to this at all beyond Crass which is seemingly quite hard to find now especially to get it working. Is there anything more up to date you guys can recommend to read up on?

What is the title of the game.

[Martinch]

Hello! I'm pretty new to visual novel extraction, but I've been trying to work my way around somehow converting the CMV video files that are part of Purple Software games (like Hapymaher/Chrono Clock) to another format. If anyone has any clues how could I do that, I'd be very grateful!

 

Edit: Turns out it was possible using crass and a specific cui made by zeas.

Edited December 10, 2021 by Martinch

[Tester]

Once more hath I created a new tool. This one is not so big as previous ones, but still very important.

There is Silky Engine, on which where are many interesting visual novels. Some time ago I hath created a script editing tool for it's mes scripts and wrote an article of it... And all the time I through there were good enough tools for it. But I checked and... Some tools can just extract data from it, other (some tool from regonme) can pack, but, alas, without compressing files per LZSS implementation, used in Silky Engine. Therefore files was getting too big and there was no method to create actually normal enough Silky's arc archives.

But that time hath come to the end. I hath tested this LZSS implementation and created SilkyArcTool, by using which thou can both extract and pack Silky archives, using this LZSS compression algorithm (I hath implemented not only decompression, as in many tools, but also a compression algorithm).

Therefore Silky Engine's (and Ai6WIN/Ai5WIN's (probably) .arc archives has fallen completely.

===

Updated: AI6WIN (and probably AI5WIN) .arc not supported. Still I have already hacked AI6WIN .arc archives, so tool will be released soon.

Edited December 23, 2021 by Tester
Tested.

[Tester]

Okay, also hacked .arc archives AI6WIN -- predecessor of Silky Engine. For them you can use AI6WINArcTool. No more thou need to suffer, packing it with raw files!

If I feel like it, I'll also hack AI5WIN's.

Edited December 23, 2021 by Tester

[Tester]

Now AI5WIN's, father of AI6WIN and grandfather of Silky Engine, .arc archives had also fallen. Alas, I hath nor archives of ancient AI1-AI4 engines, nor the means (I could find only AI5WIN ports of the games) to obtain it. Therefore I hath no choice but to stop hacking archives of Elf-Silky archives line. But probably it is not the big deal, since games on earlier engines does not work on modern operating systems.

Here is the tool -- AI5WINArcTool. As usual, unpacking and packing with correct (de)compression at both operations.

AI5WIN's .arc probably is the most complicated from Elf-Silky archives line. Data compression and storing are like in the other archives in this series, and header not very far from AI6WIN's, but... creators of the engine decided to obfisificate header with 3 (!) different keys. Fortunately, the way they implemented it was not too problematic to deal with to the extend that I implemented system that authomaticaly hacks all three keys from any (well, there are some conditions, but I could not find any archives that does not match it) AI5WIN .arc archive (yes, even not from .exe) and therefore should work with all it's games.

 

Edited December 23, 2021 by Tester

[Haoren]

Does anyone know how to repack .fpk files properly? I tried using AE as described in this thread, however the size balloons to x10 and the VN does not run. Placing the original data.fpk file allows the game to run properly. As for the engine, I think the game is using System-C, since there is a config file with that name.

[Anonym271]

Yes, I once wrote a tool that could repack SystemC FPK files and I also implemented a compression algorithm for it. Can you tell me what game you are working on? Because I believe there are at least two versions of the file format and my program only works with one of them.

However, you could also just try it out: https://github.com/Anonym271/betterfpk. You can find an EXE in the releases section or of course clone the repository and modify it to your desires.

It's a command line tool, use it like this:

Extraction:
betterfpk.exe -e <my-archive.fpk> [output-directory]
  
Repacking:
betterfpk.exe -p <my-folder> [output-file.fpk]

It's not really fast and the compression results also aren't that great, but it should do the job I guess.

[Haoren]

12 hours ago, Anonym271 said:

 

Extraction:
betterfpk.exe -e <my-archive.fpk> [output-directory]
  
Repacking:
betterfpk.exe -p <my-folder> [output-file.fpk]

 

I'm getting an 'The input path does not exist error'.

[Anonym271]

1 hour ago, Haoren said:

I'm getting an 'The input path does not exist error'.

Hm, this really should only be thrown if you entered an invalid path. What command exactly did you run? And just in case: of course you have to leave out the brackets, so an example command would be:

betterfpk.exe -p cg cg.fpk

 

[Anonym271]

@Haoren So... did it work? If not, feel free to tell me the name of the game (or give me a sample package) and I will try to adjust the program to that version.

Edited March 3, 2022 by Anonym271

[Vincent Dawn]

I'm having some trouble trying to find character CGs for Gachinko! Bitch Club. I'm using Garbro to extract the data and I can get CGs for events, backgrounds, opening stuff (like the logo, text, etc.) and the section that records event CGs, but there are no CGs for the characters. It seems like someone was able find them since there is a character CG set on e-hentai, but I can't seem to find them myself. Has anyone else had this problem or has found a solution?

[zerostage2]

So I tried to extract some of Atelier Kaguya's game with asmodean tool and some of the sprite that get split have XY coordinate in its name, is there any program to easily merge the two image? I've been using paint.net but it takes a while

[Anonym271]

On 4/26/2022 at 12:04 PM, zerostage2 said:

So I tried to extract some of Atelier Kaguya's game with asmodean tool and some of the sprite that get split have XY coordinate in its name, is there any program to easily merge the two image? I've been using paint.net but it takes a while

Can you provide some samples? I don't think there is an automated tool for this but if the file names already contain coordinates it should be relatively easy to create a simple script that does the job.

[loam]

I'm releasing a NekoNyanDatTool update (previously known as AokanaDatTool, but renamed because several games now use this engine).

It can now (un)pack Parquet (Mobile/Unity) and Senren Banka Switch/Unity files. Both games use the "aokana_switch" format; I've added aliases for those for ease of use, "parquet_mobile" and "senren_switch" respectively.

The reason it couldn't before was because the tool tried to decode the file names as ASCII, but Parquet Mobile and Senren Banka Switch have UTF-8 paths (due to japanese filenames). This has been fixed.

Binary: https://www.mediafire.com/file/0cw55ccjp2c77cn/NekoNyanDatTool.exe/file

Source: https://www.mediafire.com/file/3cdhkgijn56t4tn/NekoNyanDatTool_26052022.7z/file

Man, the bs5 is so clean compared to whatever mess this psb shit is, fucking M2 Cheeseware... I guess I'll be busy trying to port scenes to the Switch. Also I wonder why they had to bundle in e-mote for the relatively basic subset Senren uses...

Edited May 26, 2022 by loam

[scarlet]

I try to extract file xp3 but always like this. So I try to extract other file xp3 but still error. Can someone help me please?https://drive.google.com/file/d/1pd-yOimLH-iNL9kaN6K1RmanIPuahtpw/view?usp=sharing

[Kurobuns]

Hi. I've been trying to extract the assets from the PS2 version of Lamune, and I could use some help. This version of the game has exclusive content (new routes with new heroines, CG, and I think also new music) that never made it back to PC, and I really want to extract and preserve that.

 

I made a thread on the Zenhax forums previously, and I've been able to find out that the engine is by HuneX, and I've been able to extract data from the main DATA.BIN file, which includes raw videos and the rest of the data packaged in .dat files. Sadly, I haven't had luck extracting anything from those .dat files. 

 

It seems zoom909 worked on this project before, but sadly I don't think any tools were shared for extracting these assets, if he ever found a way.

 

I tried the recent PCSX2 graphics dumping tools but sadly it wasn't able to dump the main assets properly. It just outputs images in tiny chunks, and in a way that I don't know if it can be properly put back together.

 

Here's a google drive folder with the original DATA.BIN file and the extracted .dat files and movie files, in case someone wants to try poking at it. Any help would be super appreciated.

[ThugzillaDev]

On 3/28/2019 at 9:31 PM, pmrowla said:

I've been working on a tool for extracting and patching LiveMaker/LiveNovel games and figured I should post it here. It started out as a fork of tinfoil's irl which I guess was kickstarted and then abandoned a while back?

Current status is:

• LiveMaker assets can be extracted from .exe or .dat files
• LiveMaker's binary .lsb format for their "chart" scripting can be dumped as text or xml
• LiveNovel scripts can be extracted from LiveMaker's binary .lsb format
• (Translated) LiveNovel scripts can be compiled and inserted back into a .lsb file
• Modified .lsb's can be patched back into a livemaker game and run with (cp932 encoded) english text, but currently the application will lock up when you try to exit.

My current code is here: https://github.com/pmrowla/pylivemaker/tree/wip (you have to use the wip branch link since I'm not pushing anything into master until I figure out the crash on exit thing)

The code itself is mostly fully commented but I haven't published any of the actual docs yet, so there's a github issue w/a summary of what works right now and what doesn't: https://github.com/pmrowla/pylivemaker/issues/1

 

Honestly idk if there's even any livemaker games worth patching but I was bored and felt like reversing something so I picked an obscure engine that afaik hasn't been dug into very much before. I think GARbro supports viewing .gal CG's from LiveMaker archives, but that doesn't cover the text extraction and patching side of things.

Hello! I wanted to extract the visual novel "Will Melt": 

It uses Livemaker. I downloaded your github package, but I'm having trouble using it for the game's .exe or .dll. Is there a specific file in the pylivemaker I'm supposed to use? I'm very new to this so any help is appreciated

[pmrowla]

53 minutes ago, ThugzillaDev said:

Hello! I wanted to extract the visual novel "Will Melt": 

It uses Livemaker. I downloaded your github package, but I'm having trouble using it for the game's .exe or .dll. Is there a specific file in the pylivemaker I'm supposed to use? I'm very new to this so any help is appreciated

answered in PMs, but for anyone else w/this question, you have to install Python 3 first, and then install pylivemaker from pip (you don't need to download the github repo unless you want to make changes to the pylm code).

https://pylivemaker.readthedocs.io/en/latest/installation.html

[SwitchStation3]

On 5/26/2022 at 12:27 PM, loam said:

I'm releasing a NekoNyanDatTool update (previously known as AokanaDatTool, but renamed because several games now use this engine).

It can now (un)pack Parquet (Mobile/Unity) and Senren Banka Switch/Unity files. Both games use the "aokana_switch" format; I've added aliases for those for ease of use, "parquet_mobile" and "senren_switch" respectively.

The reason it couldn't before was because the tool tried to decode the file names as ASCII, but Parquet Mobile and Senren Banka Switch have UTF-8 paths (due to japanese filenames). This has been fixed.

Binary: https://www.mediafire.com/file/0cw55ccjp2c77cn/NekoNyanDatTool.exe/file

Source: https://www.mediafire.com/file/3cdhkgijn56t4tn/NekoNyanDatTool_26052022.7z/file

Man, the bs5 is so clean compared to whatever mess this psb shit is, fucking M2 Cheeseware... I guess I'll be busy trying to port scenes to the Switch. Also I wonder why they had to bundle in e-mote for the relatively basic subset Senren uses...

Hey i'm currently doing the Senran Banka Switch version as well, had any luck on it since? Thank You very much for updating it to make it work with Switch version's .dat files!

 

P.S. why can't they just fucking use .xp3 like the PC version? tryin be different 'n shit, makes things more compilated then it should or unless these companies are fully aware of the modding scene...

 

Edit: your tool doesn't work at all, how the hell do i use it dude?!

Edited November 20, 2022 by SwitchStation3

[Ohnoalex]

Is the link for ExtractData correct?? Because it just sends me to a site about running and the specific page has a 404 error;;

[jurnamo]

Hello, I am trying to extract the sprites and music from "Touhou Imperishable Night for Busy Person" by KasugaSoft. It has a data.xp3 file and a ksgdec.tpm. I have tried using Crass and it either extracts a warning text or just outputs "Crass Runtime Error.Access Violence.All Done.", I also tried using the command line, but that also gets stuck after extracting the warning text. I have also tried GARBro, arc-unpacker, xp3viewer, and Extractdata. The times I have managed to "extract" the files, everything is encrypted and will not open. 

Any help with this would be greatly appreciated!

Edited February 8 by jurnamo

[jurnamo]

On 12/14/2022 at 8:12 AM, Ohnoalex said:

Is the link for ExtractData correct?? Because it just sends me to a site about running and the specific page has a 404 error;;

https://app.box.com/s/cm8ypc3o5r

This link worked for me.