[Request / GBC] Please append RTC data in the format BGB uses for better compatibility.

[DiabloStorm]

This is to increase save portability between things like BGB that uses a separate .rtc and .sav file

Attempting to transfer saves to or from gbe+ by changing the extension works, but in doing so screws up the RTC information each time because I assume it's baked into the save file format gbe+ creates. As I don't consider this a fully fleshed out and complete emulator, I don't primarily use this as if it is. I use it for it's IR function. I also can't find where else it might be reading RTC data from or how to point it in the right direction, the manual says my computer "system" clock but gbe+ is always wrong after converting save files in this manner.
(currently in Pokemon Crystal it thinks it's Sunday 3:39am when it's Friday 8:57am. Where is it pulling any of this info from?)

[shonumi]

The RTC data is always the last 41 bytes in an MBC3 save file for RTC compatible games. GBE+ stores the last known time the game was run, using localtime data from C++'s built-in library and then adding some MBC3 registers. To make a raw game save, I advise snipping the last few bytes in a hex editor.

Some emulators have their own formats for handling RTC data, which is why GBE+ is not compatible with them (and vice versa, they're not compatible with GBE+). There's no standard "correct" way to store RTC data, but some copy formats for popular emulators.

The goal behind GBE+ isn't to be everyone's favorite emulator, or even their everyday goto emulator. It was originally built so I could have my own emulator independent from other developers (and to an extent, other users) so a lot of the design choices are geared to my own preferences, including the current RTC format. I don't want to decouple RTC data from the raw MBC3 save given how easy it is to do so manually. I'd rather not have to deal with loading an additional file for RTC-enabled games.

One of the other goals of GBE+, however, is to show others that it's possible to emulate obscure features and hardware. Ideally, I'd like to see other, more established emulators pick up stuff like infrared support, eventually. So if BGB or some other projects implemented infrared, that'd be great!

[DiabloStorm]

Okay, then i'm afraid I don't understand the context here:

The goal is to create a highly portable emulator using C++ and SDL, document the Game Boy's functions through clear code, and add as many enhancements (scaling filters, cheats, custom graphics) as reasonably possible. Additionally GBE+ strives to emulate everything about the Game Boy, including obscure accessories.

Anyway, as per your suggestion, would I be able to rip the data out of a BGB .RTC file and put it where the RTC info is located in gbe+ saves? I'm guessing I'd have to fix this with hex edit every single day I want to Mystery Gift with gbe+ by:

1. Renaming my game file from BGB's XYZ.sav to XYZ.gbc.sav for gbe+
2. Opening XYZ.rtc with a hex editor and pulling out the 41 bytes
3. Replacing the 41 bytes to the renamed "XYZ.gbc.sav" where you specified
4. Doing the 5 mystery gifts for the day and saving.
5. Rename XYZ.gbc.sav back to XYZ.sav for use with BGB which didn't seem to have a problem with gbe+ saving the rtc data directly to the save file so I guess I don't have to hex edit it again.

I know this is a niche use-case (seems to be the theme here though considering you support things like GBC Sewing machine)

Not gonna lie, unless I'm missing something, it sounds like a real pain in the ass to do all this every single day I'd like to do my 5 Mystery Gifts just to try and be immersive by emulating the experience of real hardware but without having all of the berry trees reset early and the Lucky Number reset multiple times on Friday in doing so.

Sure, I could be lame and just use a save editor to bypass all of this but that seemingly kind of defeats the purpose of this emulator. Don't get the wrong idea though, I still love that you've made this option exist at all to begin with. The problem is gbe+ is not as friendly or complete to use as BGB, otherwise I'd be using gbe+ exclusively. It was a difficult task just to map the control scheme I wanted to a controller, or even getting things to work properly/quickly (had to copy/paste the .ini raw text to create a new .ini in notepad that seems to be missing from the .7z just to get anything to work at all)
But, why does gbe+ think it's Sunday every time though? I assumed this was some sort of default...is the day of the week stored in RTC as well?

[shonumi]

Okay, then i'm afraid I don't understand the context here:

Here, a "highly portable" emulator in this context means:

1. Minimal dependencies. GBE+ only requires SDL2 and a C++17 compatible compiler. OpenGL 3.3 (for shaders and hardware scaling), SDL2Net (for netplay) and Qt (for the GUI) are all optional. On Linux, I disliked having to install a dozen dependencies to use emulators, so I made sure GBE+ has relatively few.

2. Cross-platform. Aside from some #ifdef bits related to GLEW for Windows and some a few lines to determine a user's $HOME or AppData folder, there is no OS-specific code.

That part of the readme, however, wasn't meant to refer to how well GBE+ may or may not play with other emulators. Rather, it's in relation to the codebase. The two above principles help it build easily on platforms that I don't test on often or at all (OS X, RPi, BSD). Although, I admit, the bit about enhancements is outdated, as it's no longer a focus for GBE+.

Anyway, as per your suggestion, would I be able to rip the data out of a BGB .RTC file and put it where the RTC info is located in gbe+ saves?

As mentioned earlier, the format of the RTC data/info is not the same for BGB and GBE+. BGB and VBA-M use a 44 or 48-byte RTC save format described here, which is very, very different from what GBE+ uses. On a byte-for-byte level, they're not compatible, so it's not possible to rip bytes from one save and place them in another. Rather, you would need to completely convert one format to another.

It was a difficult task just to map the control scheme I wanted to a controller, or even getting things to work properly/quickly (had to copy/paste the .ini raw text to create a new .ini in notepad that seems to be missing from the .7z just to get anything to work at all)

Yeah, that was an oversight when releasing 1.6. The .ini file was absent and I never got around to updating the release files. 1.7 is literally a couple of weeks away so, that will be rectified soon enough. Eventually (probably soon-ish), GBE+ will generate a default .ini file if it can't find one. This is something I personally want to see as well (moving copies of GBE+ around to new folders for testing purposes).

But, why does gbe+ think it's Sunday every time though? I assumed this was some sort of default...is the day of the week stored in RTC as well?

There might be something off in 1.6 with RTC handling. I can verify it is fine in 1.5. I know for a fact it was very recently broken in fde1cc5, however, I've since fixed and verified everything as of Wednesday (haven't pushed that commit yet).

Anyway, while I am unwilling to separate the raw save data and the RTC data, I am willing to change the RTC format that GBE+ uses to the same one I linked to on BGB's website. As I said, VBA-M uses that, as does mGBA and several others. The format I came up with is from well over 10 years ago (back in the Bad Old Days of Game Boy documentation), so I just came up with something on my own. I'm not particularly attached to it. I believe BGB should be able to accept those kinds of saves as-is if the format matches what it expects, therefore, it wouldn't be necessary to separate the raw cartridge RAM from the RTC data.

It also seems like that .RTC file might be some custom metadata only BGB uses internally or something. If you look at it, it's quite small (12 bytes), not enough information to accurately store all the stuff you'd need to handle the MBC3's clock. It looks like with BGB, you can copy saves from something like VBA-M or mGBA (I tried the latter) do whatever you want in BGB, save the game in BGB, then import it back into mGBA with no complications. So the separate .RTC file isn't necessary even for other emulators. It appears BGB supports appending the RTC data to the raw save just fine. It just comes down to the format used, which is something I'm interested in changing. If you want, you can rename this issue (or close it and make a new one) about supporting a different RTC format. In any case, I can get to work on it soon (and it definitely would make it into 1.7).

[DiabloStorm]

Thank you for the time and explanations. BTW, if you're wondering, on Reddit that DM was from me by the same name. I don't know if it's of any use though, as I don't know anywhere near as much about this as you.

I believe BGB should be able to accept those kinds of saves as-is if the format matches what it expects, therefore, it wouldn't be necessary to separate the raw cartridge RAM from the RTC data.

That would be wonderful. As a side note, I don't mind (as much) having to rename the file extension (would be great if BGB and gbe+ could read from the same file though), but what made you choose ".gbc.sav" ?

It also seems like that .RTC file might be some custom metadata only BGB uses internally or something. If you look at it, it's quite small (12 bytes)

Yeah...well now I feel silly not noticing that.

It appears BGB supports appending the RTC data to the raw save just fine. It just comes down to the format used, which is something I'm interested in changing. If you want, you can rename this issue (or close it and make a new one) about supporting a different RTC format. In any case, I can get to work on it soon (and it definitely would make it into 1.7).

Will do. Thank you.

[DiabloStorm]

I should clarify something in my first post. (may or may not matter) When I change the file extension to work with gbe+ the day and time are wrong, but when I change back to use with BGB
(with this activated )

Everything is fine again in BGB (day and time is correct...maybe a few minutes off at most afterward)....but all of the berry trees and other RTC events for that day have completely reset (presumably from saving the game in gbe+) But I take it this should no longer be an issue in 1.7

[shonumi]

Thank you for the time and explanations. BTW, if you're wondering, on Reddit that DM was from me by the same name. I don't know if it's of any use though, as I don't know anywhere near as much about this as you.

No problem! I'm always open to talking about this kind of stuff. Unfortunately, I haven't kept up with my inbox (Reddit, email, etc). For the past two months I've basically been in a cave, trying to finish my book. It just got published this week so I'm just now getting a chance to respond to all sorts of stuff. If I missed your DM, I'll be sure to get around to it soon. Sorry about that, but that last stretch of editing took a lot of effort (10% of the problem required 90% of the work...)

That would be wonderful. As a side note, I don't mind (as much) having to rename the file extension (would be great if BGB and gbe+ could read from the same file though), but what made you choose ".gbc.sav" ?

I believe that was a really, really old design choice I made (12 years ago?) when I first started. It was easier to just take ROM_FILENAME and append ".sav" rather than trying to fiddle with its file extension.

This also had the nice side effect of keeping GBE+'s save separate from other emulators. Well, it was nice for me at least at the time, since in the early days I needed to debug save files from my emulator (which could have been corrupt or incorrect) with save files from other emulators that handled saves just fine. There was the potential for the old GBE to make bad saves, so I didn't want it overwriting anyone's saves as well, unless they explicitly replaced it. Imagine losing a 50+ hour Pokemon Red save because the old GBE messed it up. Back then there were no save states either.

So, that convention ended up sticking years later. A lot of other stuff is like that in GBE+ (like DMG sound emulation) where if it worked and still works, it isn't touched, even if it isn't exactly conventional or convenient. You could blame it on youthful inexperience or just call it technical debt, I guess. Anyway, it's something I can change. I'm not attached to the ".gbc.sav" naming convention at all. So expect 1.7 to do things differently and more in-line with what everyone else does.

Everything is fine again in BGB (day and time is correct...maybe a few minutes off at most afterward)....but all of the berry trees and other RTC events for that day have completely reset (presumably from saving the game in gbe+) But I take it this should no longer be an issue in 1.7

That sounds about right. Something similar happens with other emulators too. They're using that 44/48-byte RTC format. They're trying to read GBE+'s RTC data as if it were the same format as their own. It kinda sorta works, a little bit, but not quite. Once GBE+ switches formats, it shouldn't be a problem. I'll be sure to test personally too.