Dolphin Progress Report: December 2018 and January 2019

While there are a lot of notable changes to go through from the past two months, there's some notable news for the general Wii community. By the time this article is up, the Nintendo Wii Shop will be closed. Purchasing will be entirely disabled so all remaining Wii Points will be rendered useless, and even downloading of purchased games will be disabled at an undefined date in the future. While this may not seem like very big news for an emulator, Dolphin does actually support connecting to and buying games off of the Wii shop.

More distressingly, it's likely only a matter of time before the Wii Nintendo Update Servers (NUS) themselves go down. Dolphin relies on the NUS servers for installing a fully updated Wii System Menu in Dolphin. Users with unscrubbed Wii discs can rely on them as well to install the System Menu after they go down, but, depending on when the game was released, it may not be fully updated.

It's also rather disappointing that the many unique titles released on WiiWare can no longer be legally purchased by users. Say what you will about the average quality of WiiWare releases, these titles are a part of the Wii's legacy, one that is slowly being locked out to those who would want to experience them in the future.

Now that we've gotten that out of the way, we have a lot of big changes that hit over the past two months that we need to get through. The sun may be setting on Nintendo's revolutionary console, but on the emulation front we still have a long road ahead of this. We hope that everyone enjoy's this month's notable changes!

For the convenience of our Android users, we've decided to cluster up a ton of important Android changes together after the more general changes. If you're on Android 9 or are a big fan of Paper Mario, you definitely won't want to skip out.

Notable Changes

5.0-9217 - Implement Resource Packs by spycrab

With the feature to replace textures in games with custom textures, users have crafted extensive and beautiful texture packs for some of their favorite games throughout the years. While there have been a few features to make creating texture packs easier in the past, there hasn't really been much in the way to change how you use them. The process is actually rather annoying, forcing users to manually extract them to the correct folders and enable a global setting in Dolphin to actually search for them. After all, these texture packs were just specifically named files in folders with no other infrastructure whatsoever.

spycrab decided enough was enough in December and introduced his plan for Resource Packs. What are resource packs? They're a new format you can use for texture packs that allow users to easily install and manage their custom texture packs. To install a resource pack, just drag the downloaded zip onto a version of Dolphin that is 5.0-9217 or newer! Installed resource packs can be managed in the new Resource Pack Manager added to the tools menu. From here, you can manually install resource packs, uninstall resource packs, and choose which resource pack you want to use for a particular game. These resource packs can also include details about what they are, including important details such as their name, version number, and creator information!

This is so much easier than dealing with files in a file manager!

This probably sounds great, but there is one rather annoying limitation. Most currently available texture packs were created before resource packs were announced and do not support the new format. Thankfully, standard texture packs can still be used and installed through the old method as this new format is more widely adopted.

5.0-9232 - Report DTK Audio in Increments of 0x8000 by booto

The Tony Hawk's Pro Skater craze of the late 90s and early 2000s gave way to a lot of incredibly popular and fun extreme sports titles. Alongside them would come a lot of spinoffs and clones that copied the formula for varying success. At the surface, Aggressive Inline appears yet to be a fairly average clone with a poorly balanced scoring system and tons of cheesy dialogue. This game would have gone mostly unknown to Dolphin developers except fans of the title were very insistent that there was a crash in this game that was the fault of the emulator.

Like the Tony Hawk's Pro Skater games, Aggressive Inline uses a feature known as Disc TracK (DTK) audio in order to stream aggressively 90s licensed music and handle voice acting. For the longest time DTK audio caused all manner of problems, but thanks to many developers, DTK audio issues were thought to be solved. Until Aggressive Inline turned up, that is.

Long time Dolphin developer booto had long assumed that he was free of the DTK curse, with issues having vanished since the early 4.0 era. Still, with the threat of DTK perfection on the line, he decided to investigate the Aggressive Inline crash to see if it really be breaking something that seems to work for every single game. This game... has some rather interesting qualities. When testing it, we found tons of game bugs, exploits, and other issues that allowed us to easily breeze through most challenges. These qualities extend to the underlying engine, as it relies on a behavior we didn't even know existed!

When transitioning between certain voice tracks and background music, the game would unceremoniously crash in Dolphin, but, the same scene worked on console. Poking around with debug tools, we were almost certain that this behavior should crash on console as well. It was only after looking into it more closely that we realized the error. booto realized that Dolphin was too precisely reporting the DTK audio position. Because the size of a DVD sector is 0x8000, it makes some sense that DTK audio would only be reported with a precision of 0x8000.

Ironically the last mission you could play was given out by a tombstone. Maybe they planned this...

Once booto added this behavior, the game no longer crashed on the afflicted scenes and was now able to be completed from beginning to end! Aggressive Inline's fan(s?) can finally celebrate the end of this very strange bug. This one is for you, enjoy.

5.0-9291 - Automatically Sync Wii Remote Extensions on Netplay by techjar

The battle to actually make Wii Remotes work on netplay has been raging for many years. Since its addition before 5.0, removal for the 5.0 release, and then re-emergence in the early 5.0 dev builds, Wii Remote netplay has slowly become easier to use. There's probably been over a dozen changes just targeting the various insanities that come from trying to make emulated Wii Remotes deterministic on netplay, and this one should hopefully be the one that finally makes this feature work for general users.

Dolphin now automatically syncs which attachments player has chosen for their Wii Remotes meaning that users now only have to be worried about configuring their Wii Remote and not worrying about what extensions are on all of the other ones. For games like Dokapon Kingdom which only require one Wii Remote connected, it should just work. It will also sync extensions on multiple Wii Remotes, but, unlike GameCube Controllers, clients may have to configure the secondary Wii Remotes inputs.

Mini rant: Dokapon Kingdom can be fully played with just GameCube controllers, but for some reason, the game will throw up a gigantic "Please connect a Wii Remote" message if a Wii Remote isn't connected. Once the Wii Remote is connected you're free to completely ignore it and use GameCube controllers to set everything up. Thankfully, once you're in game the Wii Remote can be disconnected and ignored, but still. WHY.

That said, there are still a lot of confusing and downright annoying behaviors in Wii Remote netplay. And even with this solved, there's still a pretty decent chance that any deviation for the expected use-cases will unceremoniously crash your netplay experience. Still, in general testing the feature is now working and stable without any issues if you're just trying to play games and don't try to break anything.

5.0-9271 and 5.0-9273 - HW: DK Bongos clap fix and cleanup by Billiard

While a lot of people in the Super Smash Bros. Melee may argue about tiers, characters, stages, and more, there is one thing everyone agrees on. The DK Bongos are the ultimate controller for high level Melee play. It's so overpowered that high level players have come to a gentleman's agreement not to use it, hence why you don't see professional players lugging these controllers to the major tournaments.

It's a good thing this is the case, as DK Bongos through Passthrough have been malfunctioning in Dolphin for quite some time. During a cleanup years ago, a slight mistake caused the "clapping" input to sometimes fail to work. Considering this acts as your shield in Melee and if it matters, critical functions in the other bongos games, this bug left the bongos as mostly useless. Thankfully, during another cleanup, Billiard noticed the error and fixed the bug, allowing Dolphin to work once more with the most feared controller in the Melee community.

I love the DK Bongos. They're so bad.

5.0-9279 - Qt Game List: Sort Game Titles Naturally by Billiard

Users that stay on the latest development builds may have noticed that the ordering on the gamelist has seen a radical change. Previously, when dealing with numbers, Dolphin would check "character by character" resulting in some strange ordering depending on the game. For example, Mega Man 9 would be listed after Mega Man 10 in alphabetical order. Because it was annoying, Billiard decided to fix this so that the games would show up in the correct order.

If only Mega Man 64 had a Virtual Console Release...

5.0-9279 - Qt: Revamp Controller Indicators by Billiard

Mapping a Wii Remote or GameCube Controller to other controllers can sometimes be tricky, as each controller has its own quirks. The Wii Remote has motion controls, and the GameCube Controller has notched joysticks and Analog L/R buttons with a button press at max pressure. In order to do this and tune things like sensitivity, Dolphin has a bunch of indicators and adjustments to help users understand what they're doing and get their controllers to act the way they want.

Unfortunately, the indicators left a lot to be desired, and while working on emulated Wii Remotes, Billiard jumped in to work on the indicators. The previous incarnation of the indicators were more or less ported over from Wx, and Qt gave us a lot more flexibility. With that extra flexibility, he made the indicators cleaner, easier to understand, and more colorful to help show exactly what was being changed.

The new indicators show exactly what you're mapping for Wii Remote Tilt, including deadzone regions.
Mapping Infrared (Wii Remote Pointer) is much clearer. Just be warned, some games expect you to point outside of the screen!
Adjust Input Shape so the dashed lines match your physical joystick's gate. This will allow Dolphin to compensate for different gate shapes when translating to Nintendo's octagonal joysticks. Even square gates are supported!

5.0-9447 - Replace Raster Font with Dear ImGui by stenzek

For eons, Dolphin has had the same old raster font for on-screen display messages. It's ancient, outdated, but, worked well enough that no one really saw a need to change it. If not for the fact stenzek was bored, it would still be here, but, he sought to replace it with a modernized alternative: Dear ImGui.

Rasterfont is barebones but got the job done when something needed to be displayed on the screen.
Dear Imgui has a more modern take. You can even move things around so they don't get in the way of your game!

Dear ImGui is an operating system agnostic graphical user interface that can be easily inserted into 3D pipeline enabled applications, such as Dolphin. While right now only older features have been ported over, it could be possible to do things such as improved input displays or even integrate things like netplay chat right into game window! Because of how easy it is to use, most people capable of working in C++ should be able to edit and add their own to display whatever information they'd like!

5.0-9455 - Qt: Integrate INI Editing into GUI by spycrab

Dolphin has long used an INI system for allowing developers to help automatically configure Dolphin on a per game basis. Users can also set their own defaults per game with user INIs that can even override the developer provided defaults. For a long time, we've provided a few very general settings in the Game Properties pane, but, there are literally hundreds of different per-game options out there. The only way to edit them would be to manually create or edit your own user INIs using a text editor of choice. To make this a bit easier we provided buttons to open userINIs in an external text editor, but that was more often broken than working so it was of little help in practice.

spycrab revolutionizes INI editing by giving users the power to not only view the full default INIs, but to create and edit user INIs directly from the GUI, without needing an external text editor at all! In addition to this, spycrab has also provided syntax checking and added some preset options to show users how it works and ease some more common changes. In the future, we will continue to add more presets to the GUI to make it easier for users to customize their experience however they see fit.

The old checkboxes are still present for those only requiring basic adjustments.
Advanced users can use the editor to see and change nearly every single setting Dolphin offers, either manually by typing or with the presets.

Obligatory Android Segment

As has become tradition with the Progress Reports, we have a lot of Android changes. As smartphone hardware improves, a growing share of Dolphin's userbase is on Android devices. While we still recommend powerful desktop PCs for the best experience (especially for extended play sessions), the very top end Android devices paired with the latest development builds are finally starting to show the promise that the Sonicadvance1 foresaw all those years ago.

This month, many contributers banded together to fix a plethora of issues while bringing long wanted features. Please remember to be patient with performance, most Android devices simply aren't strong enough to run most games. The very top-end devices of today can run some games full-speed, but your device's governor may resist giving Dolphin all its power for battery and thermal reasons. While we will occasionally reference performance numbers, please be weary your device may not be capable of that, and even then may only be able to give that performance in short bursts. We'll leave you with a conversation from the forums.

User: ...I have to basically set my phone on fire for it to run well?

Helios747: Finally somebody understands Dolphin on Android.

5.0-9427 - Vulkan: Fix Rendering on Android 9 Adreno Drivers by stenzek

With the Android 9 (Pie) rollout came a new line of Adreno drivers. These drivers feature quite a few important fixes that make games like Mario Kart: Double Dash!! render correctly in Vulkan... in theory. Unfortunately, in practice users trying out the new drivers were greeted with a rather unfortunate sight.

The newer Adreno drivers didn't even display Dolphin's onscreen display messages!
Using renderdoc, we could see that the games were somewhat rendering beyond the darkness.

Users excited to see potential performance increases and higher compatibility were instead greeted by the Vulkan backend being completely broken. Because different devices were updated to Android 9 at different times, we slowly got more and more reports about Vulkan being broken again. Once users confirmed that downgrading to Android 8 fixed the issue, stenzek made some time to take a look at the issue. The fact that Adreno wasn't even displaying Dolphin's on-screen display messages was his major clue that this may be relatively easy to fix.

After looking at the compiler log, he found out it was spewing out nonsense that Dolphin was somehow exceeding the output component limit (Dolphin is not). After digging around to try to find out, he realized this only happens if the varyings are defined in an interface block. This is used for features like geometry shaders (for emulating Line-width/point-size) which these drivers don't support anyway.

Stenzek's fix was to generate varyings incrementally and as needed, avoiding this driver bug.

5.0-9322 - Fix Bounding Box on GLES by zackhow

Most of the time when a game is broken on OpenGL ES and not OpenGL, it's because of driver bugs on the various Android devices. In this particular case though, the fault falls squarely onto Dolphin as it was doing something incredibly foolish.

Paper Mario: The Thousand Year Door is well regarded and beloved GameCube game full of charming characters in an interesting world. It is also a complete pain the ass to emulate thanks to using a feature known as Bounding Box as the basis for many of it's fancy paper stylized effects. It's been an absolute struggle to emulate them at reasonable speeds on powerful desktop computers, so when users reported that non-Shield T.V. Android devices were crashing on Bounding Box games, we confidently swept it under the rug with the other driver bugs.

When Bounding Box is not emulated, the game is missing many effects and will even crash at points.
When rendered correctly, it may be hard to spot Mario in his thin paper form.

So why were most Android devices struggling? Well, the first thing to remember is that the NVIDIA Shield T.V. gives us full access to NVIDIA's Desktop OpenGL driver, which is incredibly nice and allows it to stand tall against modern Android devices despite an aging processor. Most other Android devices instead use Adreno and Mali chips, which only use GL ES. GL ES is a stripped down version of OpenGL that drops a lot of legacy features. While some of those legacy features are useful for emulation, Dolphin can mostly get by with OpenGL ES 3.

This leads us to a logic bug in Dolphin - we were accidentally using glGetBufferSubData when reading pixels from the screen in GL ES mode. glGetBufferSubData isn't available on GL ES. This caused Dolphin to crash in rather sudden fashion. Paper Mario: The Thousand Year Door has been crashing all this time because of our mistake. The really sad part of this is that Dolphin already has an alternative way of reading the screen available in GL ES! To fix Paper Mario: The Thousand Year Door and the other Bounding Box games... we just had to tell Dolphin to use it. Oops.

The various Bounding Box effects even render correctly on Android, though, some are slower than others.

5.0-9424 and 5.0-9429 - Android: Add Touchscreen Support for Wii Remote Pointer by zackhow

The basic premise of this feature isn't that crazy - Wii games use the Wii Remote's Infrared sensor to allow you to point at the screen. In theory, this should map pretty nicely to a phone's touchscreen and allow people more natural control over games that occasionally use the pointer. zackhow even went through the extra effort of tuning some popular games like Super Mario Galaxy so that the touchscreen will map 1:1 with the in-game pointer position! Unfortunately, a rather strange bug stalled him during testing.

While everything worked, the Wii Remote thought it was upside down!

This was only the beginning of a rabbithole that features bugs, broken inputs, crashes, and other delays as bugs were unveiled simply by adding a new feature on top of a broken foundation. After researching more into how Wii Remotes were supposed to work, zackhow slowly pieced together how things worked. After fixing a critical bug with input handling on Android and touching things up, the feature finally worked.

Before you break out Link's Crossbow Training to celebrate, there is a significant limitation to this new feature: shooters. Most shooter style games on the Wii mimic keyboard and mouse controls, demanding constant pointer input and complex simultaneous button presses. Most of them will even literally stop the game if the Wii Remote isn't pointed at the screen! This control style was a natural fit for Wii Remotes on a TV, but on phones, not only will your fingers constantly be blocking the screen, but it also demands very unpleasant finger contortions as you press 5-6 inputs at the same time. Any game that works in this way, like Metroid Prime 3 and The Conduit, will be a poor experience with touchscreens. Not necessarily impossible, but very, very unpleasant. There isn't really anything we can do about this; these games just don't translate well to touchscreens.

Even with this limitation, this feature will improve support for tons of Wii games on Android!

Note: Due to a math error, certain phones may have issues with touchscreen Wii Remote Pointer in 5.0-9424. 5.0-9429 corrects this.

5.0-9215 - JitArm64: Fix RC calculation of srawix by degasus

Some of the most frustrating issues on Dolphin Android are because the ARMJIT isn't quite as battle tested or mature as its x86-64 counterpart. That said, it normally does a fairly good job at efficiently translating the GameCube and Wii's PPC instruction into something our ARM64 phones can chew through! In the leadup to this fix, testers were asked to go through several ARM64 only issues on the issue tracker and test out another test feature. zackhow added a JIT Debug Menu to the Android GUI, and requested testers turn off parts of the JIT to help narrow down what was causing bugs!

Testers found out that turning off JIT Integer in the debug menu, at a performance cost, fixed games like House of the Dead: Overkill, Dragon Ball Z: Sagas, F1 2002 and many others!

As Dolphin's ARM core continues to improve, JIT compatibility should begin to mirror that of desktop Dolphin. GPU drivers though...

With the scope narrowed in on a smaller target, degasus found the flawed instruction quickly and repaired it. Now all of the games work correctly without any extra performance penalty.

Note: Turning off parts of the JIT with the debug menu does not mean Dolphin is not emulating that feature, but instead makes Dolphin fall back to the much slower but more accurate interpreter for it. There are no performance improvements to be found there, as touching anything within that menu will make Dolphin slower. It is only for testing.

5.0-9224 - Android: Fix Official GameCube Nintendo Adapters by zackhow

Touchscreen controls are fine, but they can be a bit lackluster when a game requires tight controls and precise inputs. One of the coolest features in Dolphin on Android is that you can just plug in a Nintendo GameCube Adapter and use real GameCube Controllers!

...Or at least we thought you could. This feature was only partially working unbeknownst to most developers. Nintendo's officially branded adapters didn't actually work on Android, despite the fact that third party offerings had no issues. Further consider that official adapters work fine on Dolphin on desktops, and there was a bit of a mystery on our hands.

Official products tend to be more expensive than third party offerings, and many Dolphin developers didn't have the official adapter. Most reputable adapters are pretty much identical to Dolphin. zackhow already had a third party adapter, but had to specifically order an official one to try and spot the difference. Within a few hours, he realized there was a typo in the USB initialization code that most adapters seemed to ignore, but Nintendo's did not. usb_in was being used instead of usb_out in a critical spot during initialization of passthrough for Dolphin Android.

How exactly did other adapters work? Our best guess is that they're just more lenient about initialization.

5.0-9468 - Android: Fix First Party Real Wii Remotes on DolphinBars by zackhow

Your options for using Real Wii Remotes on Android are pretty limited. Android 4.2.1 ended Wii Remote connectivity with Android's Bluetooth stack, so the DolphinBar is pretty much the only solution for using Wii Remotes on Android. Unfortunately, even that had some serious issues. When using official Nintendo Wii Remotes with the DolphinBar, Dolphin would crash when booting unless you were mashing buttons while the game started. Third Party Wii Remotes were fine, leading zackhow on yet another journey into Dolphin's code.

Eventually, after checking out how several different brands of Wii Remotes worked, he figured out why official ones were causing crashes with the DolphinBar. When not hitting any buttons, usb read would error out on a read report, returning -1. Dolphin was only expecting unsigned (always positive) numbers, so when given a signed number it wrapped to an extremely large number and Dolphin started trying to read from a bunch of memory that doesn't exist. That's a segfault, so Android stepped in and terminated our process. zackhow fixed the issue by making sure Dolphin ignores negative values.

Last Month's Contributors...

Special thanks to all of the contributors that incremented Dolphin from 5.0-9180 through to 5.0-9494!


Pots continuar la discussió al fil del fòrum d'aquest article.

Entrada següent

Entrada anterior

Entrades similars