Jump to content
mnakamura

Mikan Engine: ScummVM-like HTML5 VN engine to port everything to

Recommended Posts

1 minute ago, tymmur said:

While it's interesting to add support for more and more engines, what about improving support for the already supported engines? For instance make buttons and similar, which feels like the original VNs. Once Mikan gives a decent playing experience, which isn't degraded relative to the original VN, then it would make sense to add support for more engines.

You see, my focus is not Mikan engine, but the .story format it uses. Engines come and go, but the solid interchange format is here to stay. Like, Gutenberg project collects ebooks in plain text format since 1971. How many platforms it outlived? I guess half a dozen already. Today it's HTML engine, tomorrow someone might want to do a Unity-based engine (that will still run off the same .story JSON), or a native Android/iOS engine, or whatever.

I'm not really a programmer, I'm more like a reverse engineer, so I'd be happy if someone more proficient in JavaScript took over the engine itself (and actually it's happening right now).

Work on customizable UIs proceeds, actually, Noratoto port already has an option to use UI that looks pretty similar to original.

Share this post


Link to post
Share on other sites

Oh wow, me and a few friends have been considering developing something similar for a while. Though our project would've been more similar to ScummVM's approach of providing supportive libraries and common interfaces (for virtual filesystems, configuration, multimedia...) for engine implementations that still mostly do their own thing - as supporting a wide variety of weird and wonderful engines with capabilities that range from "scripting language with native functions no more complex than ChangeBackground()" to Kirikiri by transpiling to a single generic "VN format" seems like a daunting and messy task. But hey, I guess only time will tell.

(We also would've gone for a natively implemented runtime instead of an HTML5 app, just because we feel the average VN should be able to run on a 10 years old laptop. Just a matter of pride ;))

Anyway, I haven't looked at the code in-depth yet, will check it out later, but I do have some constructive comments already:

  • (I hope I don't start a flamewar with this but) I'd recommend against using CoffeeScript, for the reasons outlined here. Don't get me wrong, I do like the language, but IMO the situation it's in gives it some significant disadvantages.
     
  • You can use things like NW.js/Electron or Cordova/PhoneGap to package your HTML5 app as a desktop or mobile application and get some APIs not available in a regular web browser (and be able to link in native libraries, or replace parts of your app itself with optimised native code). I've seen one (original English language) VN use NW.js, and some more (undemanding) games.

    For example, you could directly access the file system for loading assets.

    And maybe you'll want to use ffmpeg to decode media formats not supported by web browsers. Sure, you could compile ffmpeg to JS with Emscripten; I've no idea whether that's fast enough for any practical use, but it's definitely going to be slower (and consume more power) than a native build.

    With these tools, just build ffmpeg natively, write a binding, and use that where available, and fall back to Emscripten-built ffmpeg on platforms you don't have binaries (or the ability to load them) for.

    As far as I know, NW.js and Electron only support desktop platforms, whereas Cordova/PhoneGap was made for mobile platforms (not sure whether it supports desktop OSes these days). Not sure whether there's such a platform intended to target them all.

Share this post


Link to post
Share on other sites
9 minutes ago, DrDaxxy said:

(We also would've gone for a natively implemented runtime instead of an HTML5 app, just because we feel the average VN should be able to run on a 10 years old laptop. Just a matter of pride ;))

It's a tradeoff. Native implementation is more CPU efficient, but HTML5 makes it easier to be cross platform. Considering that devices like iPads appears to have been in mind when picking HTML5, I say it's a decent choice.

1 hour ago, DrDaxxy said:

Though our project would've been more similar to ScummVM's approach of providing supportive libraries and common interfaces (for virtual filesystems, configuration, multimedia...) for engine implementations that still mostly do their own thing

If the "universal engine" is to be truly universally used, then it should adapt to ScummVM's approach where it can read the original source files. If somebody wants to play some VN you never heard about using a supported engine, the non tech people would really prefer a guide like "copy all the arc/bin/whatever the archives are called" and that's it. The reading could then be a plug-in system where 3rd party can write plugins for new engines. The approach with first converting the VN to a new format using one piece of software and then another piece of software to actually run it seems like a user unfriendly approach.

 

Maybe a ScummVM like game selector menu could be nice. It could contain the import tool, in which case the user would feel that it's one piece of software even though it's a GUI, which actually calls multiple different pieces of software underneath.

Share this post


Link to post
Share on other sites
22 minutes ago, tymmur said:

It's a tradeoff. Native implementation is more CPU efficient, but HTML5 makes it easier to be cross platform. Considering that devices like iPads appears to have been in mind when picking HTML5, I say it's a decent choice.

Makes it easier to develop it at all, I'd say. I don't think it's a bad choice either; hell, it's probably the best if Mikan is only supposed to be the reference implementation, as OP suggested directly above my post.

 

29 minutes ago, tymmur said:

Maybe a ScummVM like game selector menu could be nice. It could contain the import tool, in which case the user would feel that it's one piece of software even though it's a GUI, which actually calls multiple different pieces of software underneath.

You could probably throw in some auto-detection magic (...which ScummVM also has) then automatically convert new titles. Though I imagine the conversion process takes a while.

Then again, a standalone conversion tool would be little more than an installer wizard from the user's perspective, except you may have to manually give it the input directory if it can't find it from a registry entry.

 

 

Share this post


Link to post
Share on other sites
22 hours ago, DrDaxxy said:

Oh wow, me and a few friends have been considering developing something similar for a while. Though our project would've been more similar to ScummVM's approach of providing supportive libraries and common interfaces (for virtual filesystems, configuration, multimedia...) for engine implementations that still mostly do their own thing - as supporting a wide variety of weird and wonderful engines with capabilities that range from "scripting language with native functions no more complex than ChangeBackground()" to Kirikiri by transpiling to a single generic "VN format" seems like a daunting and messy task. But hey, I guess only time will tell.

First of all, thank you for your kind words. It looks like the idea have been around so quite some time, I'm definitely not the first (and probably not the last :))

Both me (and half a dozen of other projects, like ViLE, Anise, RLVM, Asenheim, etc, etc) tried that road, and at least for me it's clear that it is not the way to go. The main problem with "emulate a separate VM for every VN engine" approach is the complexity. If you want to add some nice extra feature (like fancy UI, history tools, translation aid, etc, etc), you'll have to implement it multiple times, each time for an individual VM.

22 hours ago, DrDaxxy said:

(We also would've gone for a natively implemented runtime instead of an HTML5 app, just because we feel the average VN should be able to run on a 10 years old laptop. Just a matter of pride ;))

That is, again, why unified format shines. You're more than welcome to implement additional runtimes in any way you want, it is should be relatively easy (current format spec has around a dozen of commands - that's all), rather than porting all VMs and file parsing stuff everywhere.

22 hours ago, DrDaxxy said:

Anyway, I haven't looked at the code in-depth yet, will check it out later, but I do have some constructive comments already:

  • (I hope I don't start a flamewar with this but) I'd recommend against using CoffeeScript, for the reasons outlined here. Don't get me wrong, I do like the language, but IMO the situation it's in gives it some significant disadvantages.

Yeah, it's already been decided, we're moving to ES6. @Targenor is really shy and doesn't seem to like public discussions, but he's working hard on it now.

22 hours ago, DrDaxxy said:
  • You can use things like NW.js/Electron or Cordova/PhoneGap to package your HTML5 app as a desktop or mobile application and get some APIs not available in a regular web browser (and be able to link in native libraries, or replace parts of your app itself with optimised native code). I've seen one (original English language) VN use NW.js, and some more (undemanding) games.

    For example, you could directly access the file system for loading assets.

    And maybe you'll want to use ffmpeg to decode media formats not supported by web browsers. Sure, you could compile ffmpeg to JS with Emscripten; I've no idea whether that's fast enough for any practical use, but it's definitely going to be slower (and consume more power) than a native build.

    With these tools, just build ffmpeg natively, write a binding, and use that where available, and fall back to Emscripten-built ffmpeg on platforms you don't have binaries (or the ability to load them) for.

    As far as I know, NW.js and Electron only support desktop platforms, whereas Cordova/PhoneGap was made for mobile platforms (not sure whether it supports desktop OSes these days). Not sure whether there's such a platform intended to target them all.

Sure, sounds like a good idea to me. If you know NW.js or Cordova enough to implement it - you're most welcome to contribute :)

Share this post


Link to post
Share on other sites
20 hours ago, tymmur said:

It's a tradeoff. Native implementation is more CPU efficient, but HTML5 makes it easier to be cross platform. Considering that devices like iPads appears to have been in mind when picking HTML5, I say it's a decent choice.

Actually, my main point was the simplicity and largest user base. Almost everyone can read and write JS nowadays. Everyone can run HTML+JS with only a browser. JSON is a super-popular interchange format. Etc, etc.

I totally agree with those of you who proposed to dump CoffeeScript - and I agree that it was not a very good idea to begin with.

20 hours ago, tymmur said:

If the "universal engine" is to be truly universally used, then it should adapt to ScummVM's approach where it can read the original source files. If somebody wants to play some VN you never heard about using a supported engine, the non tech people would really prefer a guide like "copy all the arc/bin/whatever the archives are called" and that's it. The reading could then be a plug-in system where 3rd party can write plugins for new engines. The approach with first converting the VN to a new format using one piece of software and then another piece of software to actually run it seems like a user unfriendly approach.

Maybe a ScummVM like game selector menu could be nice. It could contain the import tool, in which case the user would feel that it's one piece of software even though it's a GUI, which actually calls multiple different pieces of software underneath.

Consider an average iOS user. Can you tell them to just "copy all the arc/bin/whaever" to their iPhone/iOS? I don't think so, 99% of them have absolutely no idea how to do that. Actually, iOS tries it really hard to make user forget what a "file" is at all. If they need any media - they have iTunes. If they need apps - they download them right from the AppStore (or whatever it's called, I keep forgetting).

Anyway, it's impossible to put a DVD into the iPhone/iPad due to lack of the drive. So, you'll need a desktop with a drive anyway. And most of the time all such things will boil down to an "installer" of some sort that will just do all that "conversion" work under the hood, while for end user it will look like a "special application that will install a VN into an iPhone/iPad/Android".

I totally agree with @Mad Pierre who pointed out that my current way to do ports (i.e. as a bunch of Unix scripts) is not really user friendly, but it's a solid base. You can always wrap GUI around the logic. It's not always possible to extract logic from GUI.

Share this post


Link to post
Share on other sites
4 minutes ago, mnakamura said:

Consider an average iOS user. Can you tell them to just "copy all the arc/bin/whaever" to their iPhone/iOS? I don't think so, 99% of them have absolutely no idea how to do that. Actually, iOS tries it really hard to make user forget what a "file" is at all. If they need any media - they have iTunes. If they need apps - they download them right from the AppStore (or whatever it's called, I keep forgetting).

Which is why that isn't a good solution either.

6 minutes ago, mnakamura said:

Anyway, it's impossible to put a DVD into the iPhone/iPad due to lack of the drive. So, you'll need a desktop with a drive anyway. And most of the time all such things will boil down to an "installer" of some sort that will just do all that "conversion" work under the hood, while for end user it will look like a "special application that will install a VN into an iPhone/iPad/Android".

It's not even all computers, which have a DVD drive today. It's big and power hungry, which is why quite a number of laptops lacks them.

 

I'm thinking some sort of solution where some sort of server is set up in windows. The user can open the browser and connect to the server and select the install dirs there and play as he likes. However since it's a server, other devices on the network can connect as well. Add some sort of "cache everything locally" for the client, then a properly made bookmark could be enough to have the VN on the go. AppStore requires Apple approving of the software, which I don't think will happen in this case, meaning a fully browser based solution would be preferred.

 

The reason I say windows is because VNs tend to be installed in windows and it would be beneficial to have them in the same OS. Making the server cross platform would be better though.

12 minutes ago, mnakamura said:

I totally agree with @Mad Pierre who pointed out that my current way to do ports (i.e. as a bunch of Unix scripts) is not really user friendly, but it's a solid base. You can always wrap GUI around the logic. It's not always possible to extract logic from GUI.

It's not that portable either because running unit scripts in windows is a hassle. I learned that if I need other people to execute something I wrote, telling them to install strawberry perl appears to be the way to go. It has a simple installer and has quite a number of unix tools. It also provides perl, which might also be useful.

Share this post


Link to post
Share on other sites
5 minutes ago, tymmur said:

It's not even all computers, which have a DVD drive today. It's big and power hungry, which is why quite a number of laptops lacks them.

True enough.

5 minutes ago, tymmur said:

I'm thinking some sort of solution where some sort of server is set up in windows. The user can open the browser and connect to the server and select the install dirs there and play as he likes. However since it's a server, other devices on the network can connect as well. Add some sort of "cache everything locally" for the client, then a properly made bookmark could be enough to have the VN on the go. AppStore requires Apple approving of the software, which I don't think will happen in this case, meaning a fully browser based solution would be preferred.

The reason I say windows is because VNs tend to be installed in windows and it would be beneficial to have them in the same OS. Making the server cross platform would be better though.

That's a good idea :) Maybe even we can go one step further: a server would not require users to actually upload their VN media files (and download converted results), but just to prove that they own it. This is where VN producer cooperation may come in really handy: if there would be a way for users to prove that they own legitimately bought copy of VN, they get their "converted for iOS" VN download right away. No uploading, no waiting, no wasting of server resources.

Share this post


Link to post
Share on other sites

There's actually a Japanese company attempting something similar (a portal for reading VNs on the web, powered by an embeddable HTML5 VN engine that's compatible with Kirikiri and allows packaging individual VNs for release as Windows/Mac/iOS/Android apps alternatively or in addition to hosting them on the web). See http://novelsphere.jp/ and http://js.novelsphere.jp/ .

The problem with having people download VNs from a (local-network or online) HTTP server into their mobile browser's caches is that browser caches are very limited. They're not gonna let you store hundreds of megabytes of assets.

I don't know what mobile data plans are like in Japan these days, but in most of the West, the traffic caused by streaming assets over the Internet would make reading VNs on the go straight from the Web impractical.

I don't think there's any way to get a generic VN reader on the iOS App Store, but perhaps there's a solution I haven't thought of.

If you want to seek VN developer cooperation, then since you're Japanese you're in a much better position to pitch your ideas to them than most people on this forum, but I still wouldn't hold my breath.

5 hours ago, mnakamura said:

That is, again, why unified format shines. You're more than welcome to implement additional runtimes in any way you want, it is should be relatively easy (current format spec has around a dozen of commands - that's all), rather than porting all VMs and file parsing stuff everywhere.

You're gonna have to put the target-engine-specific stuff somewhere - either in converter applications, or in plugins, or (like ScummVM) in an engine sub-hierarchy of the main codebase. And said main codebase could include generic functionality like a common virtual filesystem interface, a common image handling interface, implementations of common image formats (chances are you're gonna come across many titles that use JPEG for backgrounds etc.).

But then you have uncommon things like some titles (including one I'm currently working with) using 3D environments (...in my case, a single cubic room with static lighting for a few scenes in which they wanted to rotate the camera, but still, it's 3D). If you can find a way to support all this functionality with a single format, that's great, but I imagine it'll get very hairy in the long run. Meanwhile, reimplementing an individual VN engine isn't all that difficult, especially once you've done it a few times, so I'd say a bit of duplicate effort that could be saved by a more comprehensive intermediate format isn't that big of a deal.

That said (and I meant to write this in my original post but forgot), your approach is certainly more interesting than "do what everyone else who's tried this before did", and you do have a point when you say "everyone who didn't use an intermediate format failed so maybe we should try doing that instead", so I wish you luck.

For now, I'd recommend formally specifying/documenting your .story format and setting up a process/workflow to extend it for functionality it's not yet well-suited for.

Also, for your converters, you'll probably still want to write a common library providing VFS/Crypto/Multimedia/Encoding/etc. functionality. I'd even suggest settling on a parser-generator for input scripts. Hand-rolling a parser certainly isn't a bad choice, but you'll probably get tired of it if you have to hand-roll 10 of them, and want something more consistent. That would also have the added benefit that your grammar descriptions are basically specifications for all these obscure scripting languages that nobody's specified before.

Share this post


Link to post
Share on other sites

I've done some work on Kirikiri support already, and, to be honest, it's a fairly messy task :( You might have noticed that I haven't distributed any demos of games that run Kirikiri and that's for a reason ;)

Porting basically consists of 2 large steps:

  1. Unpacking of the game archives, extracting all the assets, converting images and sounds into widely recognized formats (i.e. PNG, JPG, OGG, etc)
  2. Conversion of the script into JSON .story format

I see that you've mostly completed step #1. Probably you should convert BMPs to PNGs, and that part will be done.

As for the script, typically there would be multiple .ks files. It's text-based and well-documented format, but the problem is that it is pretty low-level. To alleviate this problem, 95% of developers usually invent some form of macros (subroutines) which do high-level stuff. Take a look, for example, at this script from Imouto Ijime:

Quote

;//【SE】『ドアを開け閉め』
[ps se="se15" w=400][was]

[OnMes]

[OnName num=1]
[pv s="Vrin286"]
「ん…あれ、寝ちゃってた…お兄ちゃん?」
[cpg1]

[OnName num=0]
「ああ、鈴。ただいま…」
[cpg1]

帰ってきた部屋は電気もついてなくて暗く、鈴はソファーで小さな寝息を立てていた。
[cpg]

俺は起こさないように入ってきたけど、鈴は目を覚ました。
[cpg]

It's easy to see dialogue text (it's always plain unmarked) and ";" always marks up comments, but everything else is somewhat complicated:

  • [cpg] and [cpg1] are macros that ask for user's keypress to continue
  • [OnName ...] is a macro that displays name of a character in text box, "num=X" is a character ID
  • [pv s="..."] plays a voice file
  • [ps se="se15" w=400] is a macro that plays sound effect "se15" and waits 400 ms for its completion before continuing

And now take a look here, it's an extract from Kyoudai Hiai:

Quote

[name text = "Kaito"]
(Guess I was worried about him for nothing, huh?)
[tp]


I was about to leave his doorway to head to the stairs, when I heard some words from within. My body stiffened.
[tp]

;カットインlayer13(左イン)
[slide_cutin_hl file1="cutin001"]

[name text = "Rikuto"]
"Arisa... W-we shouldn't be doing this."
[tp]

You see, here we have completely different set of macros:

  • [name text ...] for displaying character name
  • [tp] for pause for keypress

Another example, from Oppai Sensou:

Quote

*grande_004_007_0006|
[FUNC_LOAD_PLUGIN]\
[cm][MESSAGE_ON]\
[CH_NAME_M name = "Haruto"]\
"Man, you're always such a selfish brat... Milpha, say something to her."[SYSTEM_MENU_ON]\
*grande_004_007_0007|
[FUNC_LOAD_PLUGIN]\
[cm][MESSAGE_ON]\
[CH_NAME_A name = "Milpha"]\
[VOICE voice = "mil_0327" voice_count = 0]\
"Hahaha... sowee, I'm tiwerd too... L-Let's take a little break~~ ...Ahh."[SYSTEM_MENU_ON]\

It's still the same Kirikiri, but it looks completely different:

  • [CH_NAME_M ...] and [CH_NAME_A ...] show character name, but use some extra color effects, thus two different macros
  • [VOICE voice=...] is used to play the voice file
  • [SYSTEM_MENU_ON] probably is used for keypress

In the example that you've posted it seems that:

  • [P_NAME s_cn="Subaru"] - character name
  • [P_FACE ...] used to show character's face
  • [cr] - keypress
  • [GL s_mo="1" s_fn="BG05c"] - probably a background image
  • [TL s_po="c" s_fn="SAKI_T1"] - not sure, maybe a sprite?

I have a half-written semi-customizable Kirikiri-to-.story converter written (although it's not public yet), but you still have to do separate processing of the macros for basically every KiriKiri game, as they all tend to be different (each company develops its own set of macros which they more or less copy-n-paste into each game).

@LeonidasTan, can you tell me a bit about your background? Which programming languages do you prefer, would it be ok for you to contribute to converter written in Ruby?

Share this post


Link to post
Share on other sites
1 hour ago, LeonidasTan said:

I do have basic knowledge working with python but that's mostly it. I'm sure if I do a bit if learning I could try to use ruby but its probably going to be inefficient program writing on my part.

Ok, then I propose to do the following thing: you analyze these macros and tell me what does what and I'll try to insert support for it in my converter.

I'll try to publish it soon too. I guess it won't be a problem for you to run a Ruby script?

Share this post


Link to post
Share on other sites

Yeah, they do, although sometimes it would be much easier to figure out what does what just by looking at the game in action.

Look for stuff like that in .ks files:

Quote

;//▼改ページセーブ用(ネームあり)▼
;** 改ページ時文末に使用(名前ウィンドウがあるものに使用)
[macro name=cpg1]
;履歴でボイス再生可能にする
[endhact]
;オートモード時はスキップ不可
[ws canskip=false buf=0 cond="kag.autoMode"]
[eval exp="kag.past_mess = kag.past_word"]
[eval exp="kag.past_word = ''"]
[eval exp="f.vs=1"]
[p][r]
[eval exp="kag.past_name = ''"]
[eval exp="kag.past_mess = ''"]
[eval exp="kag.past_word = ''"]
[clearFont]
[label_ani]
[stopse cond = "kag.VoiceCansel"]
[cm]
[OutName]
[endmacro]

or, alternatively:

Quote

@macro name=mruby
@ruby text=%r
@ch text=%text
@endmacro

or sometimes they even declare embedded TJS scripts and resort to calling them:

Quote

@iscript
        
;//オブジェクト生成
kag.addPlugin(global.buttonlink_plugin = new ButtonLinkPlugin());
        
class ButtonLinkPlugin extends KAGPlugin
{
    var links = []; // 選択肢オブジェクト(ButtonLinkLayer クラスのオブジェクト)の配列
        
    // コンストラクタ
    function ButtonLinkPlugin()
    {
        // スーパークラスのコンストラクタを呼び出します
        super.KAGPlugin();
    }

//...................................

@endscript

;//マクロ登録
[macro name=blink]
[eval exp="buttonlink_plugin.addLink(mp)"]
[endmacro]

Sometimes there are clearly named files like macro_*.ks (even with comments in Japanese ;)), sometimes they're jumbled together with various system stuff. If you're really unlucky, macro-like stuff can be also defined in .dlls, which makes it pretty hard to decompile them.

Share this post


Link to post
Share on other sites

Long time no news, Fuwan :)

I'd like to share a new port of Ushinawareta Rakuen - that's an ancient VN by Silky's from ~1996. Thanks to support from @Zus and @Scorp , I've got kind of a minimal playable port to Mikan engine (ripping of lots of reverse engineering details from Anise engine). Without further ado, here's what it looks like now in its 640*480 glory:

c021l1k7R9E9O4m8.png

I guess Zus's team is looking forward to translating it. So far, I've just pinned the names of the characters.

Share this post


Link to post
Share on other sites
1 hour ago, Scorp said:

@mnakamura music is supported? Means, you were able to convert it?

No animations, no music yet. Converting music is pretty trivial, though time consuming: one has to rip any OPL3 emulator (from Anise, from DosBox, Nuked OPL3, etc, etc) and render OPL3 music with it into oggs or something.

Share this post


Link to post
Share on other sites

If you are going the path of supporting old engines, then do consider adding support for old translated titles like Private Nurse and X-change. There is a number of VNs, which didn't age well due to the engine and poor support on modern computers and/or missing features. For instance I did decide to replay Private Nurse a few months ago, but stopped quickly because it turns out the keyboard can't make it move forward to next line and clicking the mouse quickly becomes annoying.

Share this post


Link to post
Share on other sites
20 hours ago, Scorp said:

@mnakamura music is supported? Means, you were able to convert it?

Got the music working, only one minor flaw remains: there's no simple concept of "looping" in these .wm files. "Looping" is implemented just as yet another jump bytecode inside these files, and, what's even worse, there's not a single stream of bytecode instructions, but several distinct streams (one per OPL3 channel), every one having distinct position & jumps inside it. Still figuring out how to calculate that looping point.

Share this post


Link to post
Share on other sites
23 hours ago, tymmur said:

If you are going the path of supporting old engines, then do consider adding support for old translated titles like Private Nurse and X-change. There is a number of VNs, which didn't age well due to the engine and poor support on modern computers and/or missing features. For instance I did decide to replay Private Nurse a few months ago, but stopped quickly because it turns out the keyboard can't make it move forward to next line and clicking the mouse quickly becomes annoying.

Ok, added it to my (pretty long) backlog :) Do they both (Private Nurse and X-change) use the same engine?

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now


  • Recently Browsing   0 members

    No registered users viewing this page.

×