Tracking and Discovering Visual Novels

[wraithseeker]

@Dergonu I'm in the process of adding official purchase links where they could buy visual novels. I'm currently working on developing another feature which is to add objects to brackets where users can decide what is the best thing + commenting. Conjurer's bracket is the raw form of what I'm developing. 

 

Hey everyone! It's been awhile and I have more free time to work on the site now and here's a list of the more important changes that I have done recently.

1.  Ajaxifying adding visual novels to your list (favourite,status,rate).
2.  Searching for visual novels now show an image and clicking on it brings you to the page.
3.  Added a VN API" https://visualnoveler.com/api/vns/search/:search where :search is the query that you are making which will return a JSON object. Example Search: https://visualnoveler.com/api/vns/search/sharin%20no%20kuni%20himawari
4. You can add notes to the visual novel library and *TO BE DONE* add a custom full screen background to your library.
5. Hovering over favourites / related VNs now display the name of the visual novel.
6. Pages are mostly mobile responsive now.
7. You can login / create accounts from Google+ or Facebook
8. Improved display of the visual novel pages.

 

 

[wraithseeker]

A pretty decent update I think? VNTLS integration coming up soon.

General Changes

1. Redesigned visual novel posters
2. User biography now uses markdown syntax
3. Improved user profile view
4. Improved Autocomplete search functionality
5. Selecting visual novel results after autocomplete will now take you directly to the page
6. Forum and main website profiles are now synchronised when you log out and login
7. Visual Novel advanced search genres are now sorted by name
8. Add page to browse all reviews for a visual novel

Visual novel library

1. Add library background images. Their positions can be customized
2. Default Tab for library is now playing
3. All section in library is now sorted by name and status
4. Added Backlog status
5. You can now delete visual novels off your list by going to the visual novel page
6. Average computed rating for Playing,Completed,Favourites and All below your library

[Scorp]

But VNTLS is a dead and was supposed to be closed in 2 monthes?

 

[DarkZedge]

Well I personally like the cleaner and easier to navigate style and the covers going along with each title as it is usefull if you don't happen to remember a title but you do recall however it's art or some part of the cover.

I hope all goes well for you guys and your project 

[wraithseeker]

1 hour ago, Scorp said:

But VNTLS is a dead and was supposed to be closed in 2 monthes?

 

I reached out to the developer and they mentioned that they would be giving me access to VNTLS's data. The hard part is probably to update the weekly translation status into the website.

[Scorp]

1 hour ago, wraithseeker said:

I reached out to the developer and they mentioned that they would be giving me access to VNTLS's data. The hard part is probably to update the weekly translation status into the website.

I mean last update of VNTLS was 2 years ago. And it will be closed in 2 years. What point to integrate something deadly outdated?

[wraithseeker]

5 hours ago, Scorp said:

I mean last update of VNTLS was 2 years ago. And it will be closed in 2 years. What point to integrate something deadly outdated?

Since the site will be shutting down and the data isn't difficult to parse, I just parse it for the visual novel archives which can be located at the visual novel page. On the other hand, the codebase for parsing data will likely be similar to as the one that parses the ongoing weekly translation status updates so it isn't that bad either.

[sanahtlig]

1 hour ago, wraithseeker said:

Since the site will be shutting down and the data isn't difficult to parse, I just parse it for the visual novel archives which can be located at the visual novel page. On the other hand, the codebase for parsing data will likely be similar to as the one that parses the ongoing weekly translation status updates so it isn't that bad either.

A better use of your time would be figuring out how to scrape VNDB to add new releases (both JP and ENG translated).

[Scorp]

30 minutes ago, sanahtlig said:

A better use of your time would be figuring out how to scrape VNDB to add new releases (both JP and ENG translated).

Do not make these ideas. vndb have an API, which he should use. Scraping the site is ultra-bad idea, both for site and for him.

[sanahtlig]

1 hour ago, Scorp said:

Do not make these ideas. vndb have an API, which he should use. Scraping the site is ultra-bad idea, both for site and for him.

I meant use the API.  Looks like the word "scrape" has a technical meaning that I didn't know (I thought it was just a generic term for extracting information from sites).

[Scorp]

48 minutes ago, sanahtlig said:

I meant use the API.  Looks like the word "scrape" has a specific meaning that I didn't know (I thought it was just a generic term for extracting information from sites).

No, it is not. https://en.wikipedia.org/wiki/Web_scraping

 

[wraithseeker]

I finally finished implementing all the translation data from 2008 to the present time. It should have a look similar to what VNTLS had in the past before they went inactive

Visual Novel Translation Status

Here's a sample of how a visual novel translation page looks like (Astral Air)

[tymmur]

I don't get the idea. What is the benefit over VNDB? There is the review section, which seems to be on the TODO list on VNDB. Now you talk about translation status, which has been mentioned on VNDB debating if it should be implemented or not.

 

You should fetch the source for VNDB and implement more features (it's open source). If you insist on making your own site, you shouldn't try to rival VNDB, but rather do what VNDB will not do. This mean implementing what has been requested on VNDB and then flat out refused. Hook codes is one of those issues. Another is VN engine (which would be useful info for translators).

 

On 22/3/2016 at 5:57 PM, Scorp said:

Do not make these ideas. vndb have an API, which he should use. Scraping the site is ultra-bad idea, both for site and for him.

Reading html from VNDB is beyond stupid. It requires a whole lot more CPU power and bandwidth for both the server and your client doing it. The API is working quite well API guide. You can usually get all the info you need and if not, you can request an update to the API. In fact being open source, you can code it yourself and submit it for review.

 

I think it would be a good idea to inform of any intension of relying on the API. It would be polite and helps track down issues if a bug causes way too much load on the server. Such problems should be solved rather than resorting to blocking due to not knowing who is causing the issue.

 

1 hour ago, wraithseeker said:

I finally finished implementing all the translation data from 2008 to the present time. It should have a look similar to what VNTLS had in the past before they went inactive

Visual Novel Translation Status

Musumaker is completely missing from the list even though it was actually being translated even when VNTLS was active

 

I have written about the translation status in post 2. It's a link to autogenerated text: translation status

I wrote a script to verify the syntax of the VN script (highly needed due to engine issues), which mean it should run every time the translation is updated. While reviewing every single line anyway, it detects modified lines and calculates translation progress based on this. In other words it is a fully automatic system, which is more or less always up to date. Even better, it's fairly easy to modify/expand, meaning it wouldn't really be an issue for me to generate a link where you can read the translation status in whatever format you want. That is, it will display the current status without history. All you have to say is which format you want the info in and precisely which info is needed.

 

Once the format is established, it will stay that way (unless requested otherwise) even after the remake is released. The total number of lines will be altered, but other than that, it should stay the same.

[wraithseeker]

11 hours ago, tymmur said:

I don't get the idea. What is the benefit over VNDB? There is the review section, which seems to be on the TODO list on VNDB. Now you talk about translation status, which has been mentioned on VNDB debating if it should be implemented or not.

 

You should fetch the source for VNDB and implement more features (it's open source). If you insist on making your own site, you shouldn't try to rival VNDB, but rather do what VNDB will not do. This mean implementing what has been requested on VNDB and then flat out refused. Hook codes is one of those issues. Another is VN engine (which would be useful info for translators).

 

Reading html from VNDB is beyond stupid. It requires a whole lot more CPU power and bandwidth for both the server and your client doing it. The API is working quite well API guide. You can usually get all the info you need and if not, you can request an update to the API. In fact being open source, you can code it yourself and submit it for review.

 

I think it would be a good idea to inform of any intension of relying on the API. It would be polite and helps track down issues if a bug causes way too much load on the server. Such problems should be solved rather than resorting to blocking due to not knowing who is causing the issue.

 

Musumaker is completely missing from the list even though it was actually being translated even when VNTLS was active

 

I have written about the translation status in post 2. It's a link to autogenerated text: translation status

I wrote a script to verify the syntax of the VN script (highly needed due to engine issues), which mean it should run every time the translation is updated. While reviewing every single line anyway, it detects modified lines and calculates translation progress based on this. In other words it is a fully automatic system, which is more or less always up to date. Even better, it's fairly easy to modify/expand, meaning it wouldn't really be an issue for me to generate a link where you can read the translation status in whatever format you want. That is, it will display the current status without history. All you have to say is which format you want the info in and precisely which info is needed.

 

Once the format is established, it will stay that way (unless requested otherwise) even after the remake is released. The total number of lines will be altered, but other than that, it should stay the same.

You do know that both websites are developed using totally different frameworks? It isn't as simple as just making a fork and editing stuff. VNDB is designed to be a database while Visualnoveler is designed more as a webapp.

As for the translation status, most of the updates are coming from the reddit translation status update threads.

I did a search for musumaker in VNTLS and I didn't see any results.

 

[tymmur]

6 hours ago, wraithseeker said:

You do know that both websites are developed using totally different frameworks? It isn't as simple as just making a fork and editing stuff. VNDB is designed to be a database while Visualnoveler is designed more as a webapp.

Are you saying you don't have any sort of database underneath the interface? If that is the case, then I predict it will become a maintenance nightmare. What you are essentially talking about is a database of translation statuses. VNDB seems perfect for this as it is based on progresSQL. I'm not an expert on webapps, but I find it hard to believe they can beat databases on database tasks.

 

If I understand you correctly, you are more or less just drag-n-drop elements into the page and then manually filling them out and this mean manually making the page for each VN. It sure sounds like a lot of work.

 

7 hours ago, wraithseeker said:

As for the translation status, most of the updates are coming from the reddit translation status update threads.

I did a search for musumaker in VNTLS and I didn't see any results.

I know it's not there. For some reason it never made it into VNTLS. It's also listed as dead in the translation overview thread, despite my reply telling otherwise. I don't know why Musumaker suffers from this issue, but it seems quite hard to tell that the translation progresses.

 

My point wasn't what was on VNTLS. It would be outdated anyway. My point is I can autogenerate a file with the current translation status and update it sometimes multiple times daily. If your page includes that file, then you will always have an up to date status without anybody updating manually. I just don't know what format such an included file should have, which is why I'm asking which format would suit you.

[wraithseeker]

4 hours ago, tymmur said:

Are you saying you don't have any sort of database underneath the interface? If that is the case, then I predict it will become a maintenance nightmare. What you are essentially talking about is a database of translation statuses. VNDB seems perfect for this as it is based on progresSQL. I'm not an expert on webapps, but I find it hard to believe they can beat databases on database tasks.

 

If I understand you correctly, you are more or less just drag-n-drop elements into the page and then manually filling them out and this mean manually making the page for each VN. It sure sounds like a lot of work.

 

I know it's not there. For some reason it never made it into VNTLS. It's also listed as dead in the translation overview thread, despite my reply telling otherwise. I don't know why Musumaker suffers from this issue, but it seems quite hard to tell that the translation progresses.

 

My point wasn't what was on VNTLS. It would be outdated anyway. My point is I can autogenerate a file with the current translation status and update it sometimes multiple times daily. If your page includes that file, then you will always have an up to date status without anybody updating manually. I just don't know what format such an included file should have, which is why I'm asking which format would suit you.

There is a database underneath the application and it's using postgresql. I think you are confused about the whole thing. What I am describing about is Web Frameworks and also here.

 

I also updated Musumaker here VNTLS status here.

[wraithseeker]

Haven't got a clue as to what features to update or add. I only updated the advanced search feature.

Feeling a real itch to do some work on the site but can't figure out much.