Something that's been bothering me for a while is where the devs moved the texture definition. In MML1 there was a definition for every draw-call that defined both the pallet and image locations in the framebuffer to use when rendering a primitive. In MML2 that texture definition has been moved out of the draw calls into a different part of the mesh definition and I've been testing over the last few days to try and find where exactly that is.
I've decided to use the chest for testing. As it's a pretty simple mesh while still having two bones and I can save state in a room and reload having a stationary mesh to observe as opposed to trying to mesh around with an enemy or otherwise potentially troublesome object. So I have the mesh definition mapped out for the most part here: docs.google.com/spreadsheets/d/1I_emUpDRFYAl46s5XVjhxf3AsAFoCNZc45CZD-FurNc/edit?usp=sharing.
One thing I found exceptionally weird is the orange part at the bottom of the mesh definition. It has a pretty constant pattern of having three byte values followed by a zero. Potential uses weren7t coming to mind, so I edited out the values to zero and:
Oh god, please don't tell me that the textures are mapped one per each face. Not to mention that the definition wouldn't even make sense for referencing textures at all. After a little further investigation i noticed something weird:
Editing part of the chest resulted in a fade pattern. If it were the texture definition I would expect each triangle to be independent of the others. Then I realized that MML1 used the reference to different pallets to simulate lighting effects, something that seemed absent in MML2's textures at first glance. And after checking some more.
I think it's safe to say that it's brightness. Which kind of left me wondering where the textures were, so I continued to edit out areas of the mesh definition I hadn't mapped out yet. And
Omg, finally. And then just to make sure I grabbed a value from another mesh in the same area and replaced it.
Yep. Definitely seems to be working. So
There it is. The weird part about it is that it's just one entry. For the chest that makes sense, since the chest is only one image/pallet. But for Roll and a lot of other characters, the devs used a trick where they use multiple passes of different pallets over the same image to draw different parts of the body while only using 16 colors per pallet at a time. So I need to check more characters to see if they only have one dword for the texture definition. And if so, then figure out how each dword is being referenced for each draw call. I also need to mix and match other texture definitions to see if it's one word for pallet and one word for image like it was in MML1.
And after that I'll have to break down the individual bits to see if they are shifted and OR'd together like they were in MML1. And hopefully this time they will have a f'ing Y image framebuffer value so I don't have to write 3,000 lines of code to manually map the appropriate textures to each model.
Rants aside. I now have a pretty good idea of most of the values in the header now.
1. Seems to be a reference for the number of limbs, bones, in the mesh 1 2. Pointer to start of high level of detail mesh 3. Pointer to start of med level of detail mesh 4. Pointer to start of low level of detail mesh 5. Pointer to bone list (0 is no bones) 6. Pointer to bone hierarchy definition (0 is no bones) 7. Pointer to texture definition 8. Pointer, effect not yet known/observed 9. Pointer to Brightness for high level of detail model 10. Pointer to Brightness for med level of detail model 11. Pointer to Brightness for low level of detail model 12. Effect not yet known/observed
And then I'm not sure what the remaining 0x200, 0x200, 0x200 is, expect that it seems to be on most models. [1]. For the first reference I'm not entirely sure what each byte is used for. Generally it just seems to be the number of limbs. But in MML1 there were scenes where the mesh included a refractor shard for when Megaman or Roll was holding one and it completely messed everything up. I ended up fudging it since it only affected a few models.
Oh god no. The one thing I was hoping for most was that the developers would fix the lack of a Y-index on their image framebuffer reference. But they liked their approach from MML1 and used the same mechanics. So it looks like I'll either have to figure out exactly how they do the black-magic texture mapping approach, or map everything manually (again).
If anyone wants to get into playing around with MML (or PSX in general), and is on windows, then I highly recommend using the NO$PSX emulator as it has tons of debug features. One of them is the ability to view the framebuffer. So the framebuffer is the PSX's 1MB of graphics memory, which can be visualized as a 1024x512 image. In MML, when an image is loaded into the framebuffer, it's actually divided into two parts, the pallet and the image. The pallet is stored on the left had side where the game is not being drawn. And the images are copied into the "grid" on the right. In most cases MML uses 4bit indexed images, which is why you can see the map is displayed as a 64x256 image.
When a face is rendered, the game looks up the x and y framebuffer coordinates of the pallet and image. And in principal all you need to do is refer to the x and y coordinates of the pallet and image files to figure out which one to use. And this works really well with the pallet file as they match up quite well. The problem is with the image coordinates. There is an x-coordinate. But I have yet to find the Y-coordinate. And this creates the problem that there can be multiple textures in the same column of the framebuffer and it's anyone's guess as to which one is actually being accessed. So the only option is to break down the list of potential textures and manually select the correct one. But it would be a lot easier if I could just figure out how they work.
So let's take the unfortunate example of the chest in the prologue. The chest uses the dword for looking up it's pallet and image of: 0b004c3c. And the chest will display correctly. Roll has three pallet/image pairs: 0b00883c, 0b008c3c, and 0b00903c. So can we replace the texture for a chest with the texture for Roll to see if we can put Roll's texture on a chest?
Looks like no such luck. And the reason is that 0b004c3c is the referenced used for the chest. Which is 0x0b for the image and 0x3c4c for the pallet. So with Roll. All of her images refer to 0x0b, and then we just just the pallet swap. So how does the game know which texture to use even when there are multiple models that use the same reference and yet have different images in game? I honest have no idea, it's black-freaking-magic. It might be a matter of load order or something like that, but there are cases where multiple models will reference the same texture, so that doesn't seem likely.
And just to check to make sure that I am looking at the correct location in the file for the texture. Swapping 0xb to 0xa shows Joe's character on the chest. So it looks like I need to make a list of models. The texture references they use. The textures loaded into the framebuffer for that scene. And then start swapping out values to see what comes up. And then compare values in their respective files, to see if there are any values that seem to stand out.
So how does this manifest itself? Let's take the unfortunate example of the chest in the prologue dungeon.
Finally got tired of testing via the command line, and ported my headless tools to the browser so I could test and preview models. It's pretty bare-bones at the moment, I haven't implemented textures or animations yet. Or even that many data types, right now I'm just reading the scene data used in the ruins, so it should allow to preview reaver bots and bosses.
Special thanks to Chiz, whose notes helped me get a better idea of how the game's bone hierarchy works. In some cases the game will swap out limbs, or have a character hold something in their hand, and I wrote a bunch of exceptions for MML1. Now going back over it, I have a better idea of the meshes that aren't needed, and I can simply skip over them, since I'm mostly interested in exporting the base model. And having a character hold something can be implemented outside of the game's file format at that point.
In terms of what to implement next, I'm leaning towards animation. If I'm lucky then they used the same format from MML1, and I can copy and paste most of my old code over to implement that. I can add in a .dae download button in case anyone wants to play with the meshes (even with out textures). And then after that I guess I'll have no choice but to look into textures. Need to figure out how to decompress the textures, and then I still need to figure out he magic formula that the dev's used to reference the image's framebuffer location.
As for how to use, it's like all of the other crap that I make programs I've made, click on the file input. Select all of the files in the DAT folder on the PSX disk and then use the right sidebar to select the scene, asset.
Edit:
And in case anyone asks where Joe's face is, I'm not 100% sure at this time.
This kind of goes in with how the game handles extra meshes that are appended into the model. In Joe's case, he has 14 bones (numbered 0 - 13) and above this is his bone hierarchy, but 16 primitives all together. So right now, since I don't have bones for the last two primitives, I'm ignoring them. But it looks like they are somehow related to the head mesh, so I'm going to have to look into these rules some more and see if I can get a better rule-set for how they work defined.
And here I was trying to make excuses. Found the face. I'm not 100% if this is correct, but it works so far in the grand total of two files that I've tested (w00t). When the hierarchy repeats the same number twice as in 13 12 13 0x0, it simply means that bone 13 (which is defined) is a child of bone 12 (also defined). In the case of when the number is not repeated, like in 14 0 1 0x1, it's interpreted as bone 14 (which does not exist), is a clone of the position of bone 0, and it is the child of bone 1.
On the end there seem to be bit flags, 0x01, 0x02, 0x40 and 0x80 of the one's i've seen so far. Not entirely sure, but it seems highly probably that one of their functions is to toggle the visibility of the mesh, and maybe different functions depending on the flag.
Congrats on finding Joe's face! The texture will look way better on it than on the treasure chest!
Agreed, though I still have to figure out the decompress the images. So I'll probably work on animation next.
And I think I finally figured out how to interpret this. The first bytes in the model give the number of limbs in the mesh. The game also defines a bone list of positions. I can't seem to find a specific number for where the number of bones is declared, so I take the start of the following pointer and subtract the bone pointer from that, and then divide by six (for the two byte x,y,z positions) to get the number of bones. And then there's the hierarchy which describes which limbs correspond to which bones (and their parent).
So the first entry is the root entry, and simply gives the number of hierarchy entries. For every entry after that, the format is the limb number, the bone parent index, the bone index and flags. So for the most part the limb number will be the same as the bone number for the number of bones. After that the bone will often repeat. And these are for limbs that are swapped out (like an open first, versus a closed fist), or objects that are added onto the model (like a frying pan in the hand). And the flags for these seem to be 0x01 for eyes, 0x02 for the mouth, and 0x80 for models that are toggled for visibility.
0x40 is another frequently used flag, but it doesn't seem like it affects models for the purposes of exporting so I can be lazy on that. And having a more accurate outline of the hierarchy means that I could probably go back and clean up a lot of my MML1 model code. I'm not in any hurry to jump back and do that, but it would be interesting.
For now the next obvious step seems to be to look at the animation. One thing the troubles me is in MML1 I was able to skip over the animation control entirely by reading the animation pointers and then dividing by the number of bones to get the number of frames. It would be nice to do something similar in this case, but I need to find the end of the last animation. So I may have to read the animation control to get the number of frames for each animation. Either way it would probably be easier to do another excel export and label the parts of the file so I'm not working blind. And the chest should be short and easy enough to work with again.
Putting off textures until meshes are complete, I've started dipping into animation. Looking into the animation controls, we see the following.
The format looks similar to MML1, but honestly I wasn't 100% sure on what all of the bytes were doing in that case either. In this file we see three pointers at the top of the file, which describe the number of frames for each animation. We then see a list of animation tracks, and then the flag 0xff to indicate the end of the animation, and likely 0x80 to indicate a repeating animation (though this is not present in the chest animation I am using as an example).
In the chest example we have the first animation which is simply a one frame animation (likely for the static pose). A 19 frame animation (likely for the opening animation) and then a 17 frame animation (likely for closing the chest). Since the chest goes from completely closed to the end of the hinge when opened, and then from the end of the hinge to slightly open. This likely accounts for the difference for why the opening animation would have more frames.
Next we come to the animation tracks for each animation. Right off the bat we encounter an anomaly where there are four pointers at the top of the file. I'm not sure why this is when there are only three animations defined in the animation control, and further more animations three and four are both pointing to the offset of 0x100. So I'm not sure why this is, but it doesn't stop us from looking at the animation tracks, I will assume it can be ignored for the time being.
For the animation tracks, one aspect that seems encouraging is that unlike in MML1, they actually fit to the dword boundaries this time. So if we go with the basic format we had before where first the game defines the root position, then the rotation of bone 0, followed by rotation for each bone. We can take the pointer to the start of each animation, outline it, divide by the number of tracks and find that for each bone we have 32 bit or 4 bytes for each entry. So one dword for position, one dword for bone 0, one dword for bone 1, ect. In this case since there are two bones for the the chest, one for the base and one for the lid, we see three dwords per track.
Since they're not using the same animation encoding, this means I can't recycle my code from MML1, but since MML1 was a complete mess, I might not mind taking a pass on this one so much. And since we've seen the developers use 10 bits for position, we can likely deduct that the x,y,z angles for animation are somehow using 10 bits as well. First I tried the approach in MML1 where I assumed that the 10 bits were directly associated with degrees, so 0x200 was 180 degrees and 0x100 was 0x90 degrees as a way of using 10 bits to encode between 0 and 360 degrees (since circles wrap around negative numbers aren't really needed if they are encoded for this).
Turn out no, that was definitely not correct, so Chiz was gracious enough to share his notes with me, and I decided to take a look.
And his method definitely seemed reasonable, and likely the result of a lot of testing. So I tried to implement his notes to the best of me degree where I take two bits as the magnitude, and then eight bits from each of the rotations and apply the logic that is indicated. Where the angles encoded aren't degrees directly but express a fraction which is then applied to a magnitude. And I could not get the implementation to work correctly.
So taking Chiz's notes to heart as highly plausible, I decided to do more testing and see what values the game would display when I injected values directly into it. For a testing method I created a save state in the first chest room of the prologue, and then extracted the save state with gzip. I then wrote a program that would look for the header of the static chest animation, and then overwrite the dword for either the root bone's Y-rotation or bone 1's Z-rotation, and then repack the save state with g-zip so I could quickly test with binary values.
And the results of what I found confirmed Chiz's notes with one difference. In his notes we wrote the range to be from 0x80, 0xc0, 0x00, 0x40 and 0x7f, which is why i was using 8 bits. When using ten bits the values, the tenth bit is 0x200 and the ninth bit is 0x100. So I think that's what was causing the confusion. Aside from that aspect the mechanic behind how the angles are encoded was completely correct.
From lowest significant bit to most significant bit, there are 10 bits for x rotation, 10 bits for y rotation, 10 bits for z rotation and 2 bits for magnitude. Each rotation value does not express degrees itself, it represents a fraction between -0.5 and +0.5 which is then multiplied by the magnitude to give the actual degrees to be rotated by. Magnitude has four possible values which are
0b00 = 90 0b01 = 180 0b10 = 360 0b11 = 720
Because of the fraction being multiplied by the magnitude never exceeds 0.5, this means that the range of values expressed by the each magnitude is
0b00 = 90 deg or +- 45 deg 0b01 = 180 deg or +- 90 deg 0b10 = 360 deg or +- 180 deg 0b11 = 720 deg or +- 360 deg
The way the bits work for rotation is the most significant bit is -1/2, the next bit is 1/4, the next bit is 1/8, and each bit has a very specific value. So these values are added up to give a fraction which is then multiplied by the magnitude.
In practice, it's used like the above. The most significant bit 0x200 can be set and that will move the hinge all of the way back to -45 degrees. Setting the first two bits with 0x300 will move the hinge all of the way back, and then bring it down towards the center. Not pictured is that if all of the bits are set, the hinge will be moved back towards the center. And then to go in the positive direction the last bit is left unset and setting the lower bits will rotate the positive direction. Also note that the image above is for illustrative purposes, not to indicate the direction of positive and negative.
Using this method of interpreting the angles leads to accurate results from what I can tell, and while a lot of animations are correct, others still have problems which means I need to break down the animation controls, and break down possible reasons for why these animations are displaying incorrectly.
Okay, so I managed to fix most of the previously non-working animations. I'll start with what was wrong. The header for a scene file will define a list of models. For each model the list will provide a flag to for the model, a pointer of the mesh of the model, a pointer to the animations tracks for that model (if exists) and a pointer to the animation controller for that file.
This is the animation controller for Roll's model in the prologue. There are 17 pointers to animations in the header.
This is the animation track (header) for Roll's model in the prologue. There are 14 pointers to animations in the header. The main problem was my incorrect assumption that the number of animations in the controller would match up with the number of animations for tracks in a 1:1 fashion. So I read the number of animation controllers, found the number of tracks for each controller and assumed that matched up to the same number for that animation track. This assumption was incorrect, caused my program to read crazy values outside of the animation track, and crashed for a lot of models, and provided incorrect animations for anything else. (Note that these were testing versions. Number 12 is the last one I posted a link to, I'm on number 20 atm, so if anyone is curious change the link to anything in between and observe the miserable failure).
So the way the animation tracks and controller actually work is pretty simple and somewhat anti-climactic. The conclusion that the number of animation controllers doesn't match the number of animation tracks is because multiple controllers refer to the same animation track. There is a list of pointers to animation controllers in the header. The first byte in the first dword in the animation is the animation track number. So if we see in the example above that's 0x0d or 13, which refers to a specific track. The second byte in the first dword is the length of the controller. From there for each controller entry, the first byte is the stride and the second byte is the number of frames. So in most cases this will be 0x00 1, 0x01 1, 0x02 1. So it would be interesting to edit this in game to see if I could set a longer number of frames to see if I could make the chest open animation longer or something.
If the last byte in an animation controller entry is 0xff, it will stop and 0x80 will cause it to loop. Generally these are put at the end of the animation controller length anyways, but the if the flag is written into any controller entry before that it will have the same effect of either the animation ending on that frame, or causing the animation to loop up to that frame.
So now that I have a pretty good idea of the geometry file format type, I need to go back and look over the files that work and see if there are any issues that need to be addressed. Also I'm only looking at the magic number 0x0a and when it's the first file in an archive, so I need to look around for more files with models in them. And then I need to start looking at the faces. There should be a flag for double-sided triangles. And then from there I'll have to start looking into the bits that describe which texture to use for each face, and then start looking into the textures themselves.
Test for the frame duration hypothesis for the second byte of each animation control entry. Each entry has been changed from 0x01 to 0x20 (Might be hard to read the hexidecimal since the save state is shifted over one byte). This is the result:
Test for the animation track stride hypothesis for the first byte of each animation control entry. Each entry has been changed from an incrementing stride (00, 01, 02, 03, ect) to a fixed 0x10. Meaning it will only show the 16th frame of the animation, and hold it there for the duration of the animation. Note to increase the effect of the animation, the duration has been set to the previous 0x20 frames per pose.
Test for the animation track selection hypothesis. This test reuses the edits from the first test to exaggerate the duration of each pose. The only change is that the first byte of the first dword for the animation control has been changed from 01 to 03. This should change the animation to use the chest close animation track as opposed to the animation open track, and the chest will start with a fully extended hinge and slowly close.
Since I've more or less reached the limit on what I can do in terms of geometry, it's time to shift my attention to textures. Needless to say compress is not my forte, but I'll write down everything I can make out and we'll see if I can scum my way through it. Since the content of the PSX and PC versions should be identical, I'll compare Roll's texture from the launch pad in ST08 to see what hints I can dig up. I'll start with the header on the PSX version.
The first number 0x03 is the magic number for a TIM file that includes a pallet. The second number 0x4080 is likely the extracted (full) size of the texture in bytes. The third number, 0x03 is currently unknown. It seems to be a repeat of the magic number. ※1 The third number 0x0100, 0x00f0 is likely the pallet frame buffer x and y coordinates. ※2 The fourth number 0x40, the number of colors and 0x01, the number of pallets. ※3 The fifth number 0x200 is likely the image framebuffer x and y. The sixth number 0x40 is the with and 0x80 is the height. ※4 The seventh value 0x13c is likely the length of the compression bit field.
※1 Potentially it could be an enum for the number of bits per pixel. ※2 Framebuffer x and y are likely unsigned short values, with the way the game handles the texture references I'll likely have to break these down in binary data to see how these are masked and shifted as a reference from the mesh. ※3 To be honest I think this is a lie, it's likely there are 4 pallets of 0x10 colors each. But with the way the game handles the offsets for the pallet, it probably doesn't matter how the game copies the pallets into memory as long as they can be referenced later. ※4 0x40 is the framebuffer width. Since the image is stored as encoded 4 bit values in the framebuffer, this means that the image width is 1/4 what it is in the framebuffer when compared to being rendered. This value is likely set so that the data wraps appropriately in the framebuffer.
With notes for the header out of the way, let's take a look at the PC version data.
In the first row, ignoring 11111111 we see that the pallet has a length of 0x8c with the values repeated for the framebuffer location and number of colors. The next eight lines are the pallet data as rgba5551 16 bit values. For the image we see a size of 0x400c, and the same values for the image framebuffer location and width and height. Since the pc version has a header that is always 12 bytes in length, and we see that the lengths are 0x8c and 0x400c, if we subtract the length of the header we get, 0x80 and 0x4000. If we add these together we get 0x4080, the decoded length given in the psx version. So the PSX data is likely the pallet data sitting ontop of the image data. This means I can take the pallet and image data from the PC data, and create a sample output file to compare the decompression of the psx game to.
We can take another look at the PSX data again. From the PC data we know that the pallet data starts with '00005fd3 bebedca6'. Which we see at 0x3f96c. Since the data starts at 0x3f830, if we subtract these values we get the 0x13c length given in the header of the PSX data. One thing that's striking is there doesn't seem to be any value for the size of the compressed data. Which likely indicates that the decompression reads as it goes until it reaches some termination condition. My original thinking was to cut the file data into it's own file, but I can likely include the padding up until the next file descriptor to be safe. This means that in practice I will have to read the header, copy the compression bit field into memory and then place the file pointer at the start of the data and read as I go along. So I'll go ahead and cut these files out of their respective archives, so I won't have to keep finding their offsets all of the time.
Edit:
IN hopes of finding a ready-to-use decompression I tried searching google for "playstation lzss" and came across the Chrono Compendium, which provides a description of how the algorithm works. It is a chain of 9 bit and 17 bit values. If the first but is zero, then the next eight bits are copied directly, otherwise if the first bit is 1, then the next 16 bytes represent a jump, back and copy command. Which give the location to jump back to and the number of bytes to copy.
In the case of the data, we see the yellow area at the top that's something, the blue area that's the pallet and then the purple area which is the actual data for the image. I highlighted parts of the data in orange that are different from the unpacked version. If the compression was the same one being used in Chrono Trigger, then the data would be different from the output with the extra bits that would be added in and shifted over that didn't fit with in the byte boundaries. But being familiar with the basic concept of what lzss is, we can take a look at Chiz's notes.
And Chiz's notes are pretty consistent with the Chrono Compendium notes. So it looks like the green part in the top of the file is the bit field, which describes if the data should be directly copied or not. We also have a format for how the jump back and copy words are comprised of 12 bits for the address and then four bits for the number of words to copy. So the next step is to start by looking for differences in the files, highlighting the words that are jump back and copy references. Try to find out which data they point to, and how many bytes to copy. And then go back and look at the bit field to see how many bytes each bit describes.
Edit 2:
I changed my excel export to write to words instead of dwords so I could outline the words that look like instructions. The light blue part is the pallet which is the same for both. And then I highlighted the copy and jump instructions on the left, and then highlighted the parts that are copied in on the right. I added in a red highlight before every copied section to make it easier to see the start and stop of each copied area.
There are a lot of things that confuse me. As sometime it looks like a single word is being copied multiple times, and other times it looks like multiple words are being copied. And another trend is that I can't get the bits in the bit field to match up with the distance between the words that look to be jump instructions. But for now I can at least look at the copy and jump instructions and see if I can break down their bits. One thing I keep wondering about is if the pallet is included in the compressed data or not. Since the pallet length is included in the size of the decompressed file in the header, I can only assume that it is.
Edit 3:
As for testing approach, I can pick a texture that I know doesn't get loaded until entering the first dungeon (like the chest). Then make a save state right before the dungeon, and then make another save state right after the dungeon, search through the memory and find the original texture in memory as a control. Then I can make changes to the game iso, and reload the first save state, enter the dungeon and dump the memory, and then copy the specific area of memory where the chest image data is copied to. That way I can make changes to the bit field data to see when and where changes are made if a bit is changed from 1 to 0. And I can also attempt to change the number of words copied over from previous references in the file.
Edit 4:
Stage 0F Chest position offset in iso: 0x22bcee8. Pallet Starts at in iso: 0x22bcfbc. Image Data starts at in iso: 0x22bcfbc Save state pallet offset 0x91455. Save State image offset: 0x2dbcc9
Which means I can now conduct tests by changing the game rom, loading in stage 0F, and then saving the game state and checking the memory offset to see what was impacted by my changes. In theory. It looks like with the way the game stores the framebuffer, the image data ins't one long entry, it's mixed row by row with all of the other images in the framebuffer. I can either find the stride to rebuild all of the data, or try to use the first group of collective data effectively to try and test what I can. The first thing I can test is to start with the unedited data, copy the data for the first row, then set the first value of the first bit field dword to zero and then take another data sample. That should give me the position of the first jump-back-and-copy command.
This is not what I expected at all. I was thinking that if I changed bits in the bit field from 1 to 0 that they'd copy over the data directly as opposed to execute a jump and copy command. Meaning that after the first few literal bits, the difference in the two files would be the first few jump back and copy commands. Instead the beginning of the data just got overwritten with data from later on in the file?
I get the feeling that the core mechanic is simple, I'm just too stupid to figure it out. The jump back and copy commands seem simple enough. An address, which with enough data points probably wouldn't be too hard to figure out, and then half a byte for the number of words to copy. It's also not to hard to get an idea of which words are jump back and copy commands by looking. The hardest aspect is the bit field. I can't get the 1's and 0's match up with the words that I know are jump-back and copy commands. And any attempt to edit or change the bit field results in unexpected changes. I think I'll start porting my geometry exports to the pc version files so I can start mapping textures, and then figure out what to do with the psx version later.
After not having any luck with lzss (the look back and copy instructions seem simple enough, but the bit field makes absolutely no sense), I went back to the PC version. Right now quick outline, pretty much the same as every other debug page I'm made so far, with a few minor changes. Files are now saved to IndexedDB as ArrayBuffers and I realized I can use the browser's native DataView object to read buffers directly as opposed to having to copy in my small library every time. Added an explanation to the left hand sidebar when there are no files loaded, and it takes some amount of time for the left hand bar to be populated so I added in a message to let the user know that the page is working and hasn't frozen.
Archives are lazy loaded in, when you click on one for the first time, it will be read on the spot, and the list items will be populated. After being loaded once, the unordered list will remain to be able to open again without needing to be re-parsed. Basically a lot of tiny little things, but I'm always amazed with UI programming that you can take even a relatively simple interface and find things to tweak and improve, and hopefully I'm getting closer to being better able to present the information in the game files.
Anyways, right now there's not much that works. Import files, click on an archive name to display the file contents. Right now only images are supported, so you can click and image and display it as a mesh in the 3d viewer. Next step is porting my PSX mesh files, and then starting with something simple like the chest or a horroko to test displaying with a paired mesh. And then from there start tracking down the data points I need to match up the texture / pallet pairs. Also spent sometime to fix the images that weren't loading before. Turns out the PC images are so stupidly simple as to be overlooked. Images with a pallet length of 256 are 8bpp, and anything less is 4bpp. Width is given in length of shorts, so two bytes per short = width x2 for 8bpp and width 4x for 4bpp. Otherwise all you do is read the image files in sequence and then render the x and y coordinates for each pixel.
Ported static mesh code from my psx page. One thing that's nice about the DataView object is they make it easy to open a segment of the file. So i dont have to deal with adding the global offset of a file to the internal file pointers. Next step is adding in the textures. I think I'll start with something like the horroko and pair the textures manually. Then i'll have to figure out how they are referencing the x,y locations in the framebuffer by breaking down all of the references into binary. nd then for the face I'll have to figure out which of the four bits references the texture selection and the framebuffer y offset.
Edit:
Took some time to do some house cleaning.
- Moved hosting from Digital Ocean ($60/year) to VpsCheap.Net($20/year). - Made new landing page for mml.dashgl.com - Add link for the current PC mesh project - Uploaded all of the source code to Gitlab under GPLv3 gitlab.com/users/kion-dgl/projects
Next step is to precode in the texture for a horroko and then see if I can get it to map to the mesh correctly. And then follow up to see if I can start mapping other textures.
Managed to get the textured mapped to a horroko by pairing it manually. With all of the assets in MML2, I'd rather try to have them pair automatically if I can help it. I'll use this post to write out notes for what i can gather on linking textures.
As for linking textures I can try to make a few assumptions.
- The largest texture size should be 256 x 256, so I can use the same texture size for everything
- 256x128 textures can either take up the top or bottom of a texture. I can either find the minimum v value of a texture and use that, or be lazy and draw the texture twice
- For multiple pallet/texture pairs I can use just multiple textures. Before I was trying to use the gltf exporter, which only supported one texture. if I use my own .dae exporter, then this restriction doesn't exist.
- I can make a lookup table for while archive files refer to which texture lists for searching. That way hopefully I can just pair a list of files with similar names, and not have to pair every asset individually.
Edit:
For ST0F00.DAT for models we have
1. Horokko 000a 3c10
2. Fingerii 000a 3c44 000a 3c48
3. Chest 000b 3c4c
4. Mimic 000b 3c4c
5. Joe 0008 3c50 0008 3c80 0009 3c84
6. Roll 000b 3c88 000b 3c8c 000b 3c90
Which means the first thing I need to do is make a table for the values for the list of models versus the list of images mapped to them.
Edit:
Horokko Texture Width: 256 Height: 128 Framebuffer Image X : 640 Framebuffer Image Y : 0 Framebuffer Pallet X : 256 Framebuffer Pallet Y : 240
Fingerii Texture Width: 256 Height: 128 Framebuffer Image X : 640 Framebuffer Image Y : 128 Framebuffer Pallet X : 64 Framebuffer Pallet Y : 241
Chest Texture Width: 128 Height: 128 Framebuffer Image X : 704 Framebuffer Image Y : 128 Framebuffer Pallet X : 192 Framebuffer Pallet Y : 241
Joe Texture Width: 256 Height: 256 Framebuffer Image X : 512 Framebuffer Image Y : 0 Framebuffer Pallet X : 256 Framebuffer Pallet Y : 241
Joe Face Texture Width: 256 Height: 256 Framebuffer Image X : 576 Framebuffer Image Y : 0 Framebuffer Pallet X : 64 Framebuffer Pallet Y : 242
Roll Texture Width: 256 Height: 128 Framebuffer Image X : 704 Framebuffer Image Y : 0 Framebuffer Pallet X : 128 Framebuffer Pallet Y : 242
Which means from this we know that the Fingerii and Horokko textures occupy the same slot, and Roll and the chest textures occy the same slot. And looking at their texture references, we see that the Fingerii and Horokko both have 0x000a as a reference, while the chest and Roll both have 0x000b as a reference.
0x000a is 1010 in binary and 640 is 1010000000 in binary 0x000b is 1011 in binary and 704 is 1011000000 in binary
So it basically looks like all I need to do is shift the bits up by six, which will at least give me the X coord of the images in the framebuffer. Though it seems pretty dumb the way for them to do this as they might as well encode the Y coord as well, or at least add in an extra bit because they have space left over.
Either way, let's see if the comparison stacks up. We'll try the reverse for Joe. The two 0x08 textures should be the body and 0x09 should be the face texture.
512 is 1000000000 0x08 is 1000 576 is 1001000000 0x09 is 1001
Which means we at least know where the x coordinate is. And The offset of 128 in the top or bottom of the 256x256 is defined by the uv values. Not sure if there is a bit for the top or bottom "shelf" of the framebuffer. For now it looks like all of the textures are either 0 or 128 in offset, so it could be the devs used the top row for enemies and the bottom row for the environment, and then didn't need a y coordinate.
At least I have a general idea of the image X reference to the framebuffer. Next step is to figure out how the pallets are referenced. Which will probably be largely the same for the first pallet at least, but I'm wondering if I'll needed to do anything for non-default pallets in terms of calculation. And then lastly I'll have to figure out the face reference for which pallet to use.
Though if the devs are shifting the bits up six, then that's exactly what they did in MML1, so I might as well leave this code here, so I can use it to double check the pallets.
Okay to summarize things (for myself, mostly) it looks like the X coordinate of the framebuffer is easy enough to find, and while I'm not sure about the Y coordinate, since all of the textures for enemies are less than 256 it means I don't have to worry about that until I come across data that suggests otherwise. As for rendering the textures I have two options. I can be lazy and simply draw the 256x256 square with all of the textures as it would be stored in the framebuffer, or I can try to track the minimum u and v coordinates respectively and try to match just the image that matches up with that. The way I have it set up right now is more geared towards the latter option of tracking the min-coordinates and just drawing a single texture. That should also reduce the time it takes for the texture to be rendered and the model to appear on the screen. So I'll start with that option, and then use the framebuffer as-is approach as a brute force option should trying to match a single texture not work.
The next step to focus on is the pallet. And since I know that the chest and horokko only use a single pallet / image pair for their texture, that means I can at least find the mapping relationship for how they are paired. I can then test with Joe and Roll against all of the pallets in the file to see if there is an appropriate pair for each of them.
Horokko Pallet 3c10 0011110000 010000 Framebuffer Pallet X : 256 100000000 Framebuffer Pallet Y : 240 011110000
Chest Texture 3c4c 0011110001 001100 Framebuffer Pallet X : 192 11000000 Framebuffer Pallet Y : 241 11110001
Which is good because it looks like exactly what was used in MML1. Mask the bottom six bits and shift them up by 4 bits for the pallet x coordinate. And shift six bits down for the pallet y coordinate. So now that I've confirmed the pattern, the next step is to see if the references can be calculated to match the list of pallets in the file.
Basically I was able to match the pallet x and y references from the model to a pallet given in the file. One thing that I was wondering about and may need to look into eventually (but hopefully not) is if multiple pallets are stored in each pallet. Which sounds kind of stupid, but in MML1, since the pallets were often used for lighting effects, it meant that the game only ever referred to the first entry in each set making the references easy to track down. With MML2 lighting is defined inside the model to be calculated inside the fragment shader. Which makes me wonder what's actually in the pallets, because with 16 colors (a length of 0x20 bytes), most of the pallets are declared with a length of 0x40 bytes, which means that there's enough room to fit another pallet in there. I'm happy as long as I can just calculate references and not have to mess around with finding the closest coord to calculate which pallet is being used internally, but I might manually add an offset in there to see what that extra pallet room is being used for.
That aside, I have the x reference to the image, the ability to ignore the y coordinate (at least temporarily) and a function to get the pallet x and y coordinates. That means I should be able to at least display models with a single texture, and then start looking into the extra four face bits to determine which bits are used to map textures.
Edit:
A little progress. Right now I'm only mapping one texture per mesh. The next step is to figure out how textures are referenced from the extra four bits left over in the face. There might be a more elegant way to go about doing this, but I think I can skum my way through it. I can make a list of the rendered textures and then try combinations of bits to see which ones output the correct render. Since there are only a few possible combinations, I can probably use Roll and Joe as test cases, and if they work, then it't probably the right combination, and I can move on to expanding my test cases.
I don't even know what to say. Things never "Just Work"â„¢, normally any small change requires tons of notes and testing and pain. But well, something I never actually even expected to work, actually worked. I am so confused right now.
So with the mysterious four bits on the end of each face definition, I was going to start looking into the texture reference (now that that's a thing) and I was wondering where I should start. So I thought, "well, why don't i just start with the two smallest bits and work my way up from there.". Refresh the page, and there's Roll, rendered. First try. My whole world is shattering around me.
Also added an export tab on the top of the screen. So single images will be displayed there when viewing an image directly. Textures for meshes will also automatically be populated. I don't have models there yet as I still need to add in bones and stuff before I want to put that back in there.
Right now it looks like I need to approach testing and mapping more models. Probably writing a lookup table for which file different models reference for textures. And I found some models that use textures stored in the bottom half of the framebuffer. So I need to look into how those are referenced.
Edit:
Okay it looks like there is a bit in the texture definition (not the 4 bit face) for if the framebuffer y pos is 256 or not. So basically the framebuffer is divided into 256x256 segments. The image coord gives the x and y position relative to 256, and then inside that area uv is used to determine which image to map the the model.
Next step is more testing to map to more models (or be a dirty cheat and just put a debug material on textures I can't find).
Edit:
Did some more testing. I loaded more files from MML2, and pointed each archive to a default file to look for pallets and images, which works for most but not all models. So I added the option to define a specific file for a pallet and an image respectively (in case the game does something stupid like that), but it looks like the pallet is almost always contained in the same file as the image.
Working with the framebuffer could be a lot better. Generally I'm pretty confident in the code I have to get the right 256x256 area of the framebuffer, but matching the image 1:1 is a lot of trial and error and trying to write a bunch of if statements to catch different conditions, but images just keep falling through the cracks. So right now I'm starting to lean towards drawing all images that match the framebuffer coords and see if that works any better.
But aside from a few ends and ends, overall it looks like a decent portion of the characters and reaverbots get rendered correctly, so I'll clean up and match what I can on the texture side and then move back into re-adding bones and animations.
Okay, so models are now textured rigged and animated. I have a few more tweaks that I need to add in the PC viewer page 1. Add option to clear IndexedDB/localStorage 2. Add 'thanks" and gplv3 copyright into the about 3. Add an exception to force mesh normal material when a mapped texture file isn't found
But those shouldn't take very long, so I'm going to start focusing my attention on the .dae exporter. In all honesty I can just copy and paste my exporter from the MML1 page into this page, and it will work fine, but if I did that I wouldn't bother taking the time to improve on it. So hopefully waiting a week or so to be able to export MML2 files isn't too much of a wait for everyone.
The main difference I want to implement for the change is using a xml library versus just writing the xml out as a string. Basically what I have now is "<v>" + x + "," + y + "," + z + "</v>", which even if you aren't familiar with code should at least hopefully strike you as a pretty ugly way to do things. If I use an xml library, that means I can build an object representation of the xml to be exported, and that way I have code that is a lot easier to manage.
So first things first, on the Khronos group gltf Github page, they have a repository for sample models. So the first step is to take samples, load them into blender to make sure they work. Then take the sample, parse it, convert it to a javascript object, change nothing, convert it back to an xml string, write it as a file. And then test to compare it to the original file and test if it works in Blender.
So quick note while testing, this is the overall layout of the object representation of a .dae file. { "COLLADA" : { "$" : { "xmlns" : "http://www.collada.org/2005/11/COLLADASchema", "version" : "1.4.1" } "asset" : [], "library_images" : [], "library_effects" : [], "library_materials" : [], "library_geometries" : [], "library_animations" : [], "library_controllers" : [], "library_visual_scenes" : [], "scene" : [] } }
Edit:
Okay and first test complete. Short test to load the rigged figure sample, parse it, then rebuild it then write the output. pastebin.com/waiHqNiJ. And testing it in blender shows that both the original and parsed and exported version work. Which means we can move onto the next step which is breaking down the individual pieces of the dae file, to see what kind of structures I will need to implement for each.
Edit: Okay, so far I've managed to confirm that the xml2js library is compatible and can pass assets through it and get the same thing out. I also have a general structure for the json object that can represent a dae file. Basically, everything is under the COLLADA name space, $ contains some general meta information about the schema and version. And then I mostly have to think about are the attributes the define and describe the model. So in the next step, we can cheat and use the exports from the MML1 page, pass them into the XML parser, and see what comes out. That should give us a pretty accurate idea of what arguments we need to pass into the new version, and make it work. So starting with COLLADA.asset we have:
So created [at] and up axis are pretty self explanatory. Not much to change there. I should probably pick a better name for the tool, either get more creative with the comments or leave them out entirely, and probably provide more information like the file name and file offset of the data being exported.
Edit: So looking at the other models, it looks like comments include the arguments that the model was exported with, and source data includes information about the file on the disk. In this case I probably don't have too many possible options to include in the comments (maybe the source page that was used to generate it?), and then source data I'll include the filename, type, offset, and possibly image names.
Quick rant about the .dae file format.
Dae includes everything and the kitchen sink, so you can store multiple scenes, with multiple models, cameras and lights (nodes). Which is a lot more than I really want or need to store in the file (just one mesh). So you end up having to fill in the scene data with preset values. Not too much of a hassle, but it adds an extra layer that would be easier if it wasn't there. So at the bottom of the file, in scenes I define a default scene. Then in the Library visual scenes part, I need to define a mesh, and that mesh will then point to a geometry that defines the mesh.
I was going to cheat, but I think I'll take a similar approach that I made before, where I used the sample models from the gltf repository, start with the box, use preset values and slowly modify that until I'm able to manage more use cases.
Edit:
Fission mailed. I tried making an exporter that used a hard coded box in json format, that would be converted to xml to see if it would work in Blender. No luck which means I need to try making the values closer to what they are in the original file and try again. I've got two methods of debugging available to me. I can start blender from the command line and that will show me errors when I try to import a file, and I can also format the original file and format the output file, and compare them to see if there are any differences.
Overall, what I probably won't stop complaining about is how annoying it is to work with .dae. Having multiple scenes seems like an overreach, but that's only one layer, set a default scene, and then define the scene in the visual scene area. In the visual scene area, you're defining nodes. So a node can be either a mesh, a camera, or a light, and these can all have parent-child relationships and transformations like scale, rotate and translate. Again I can ignore all of these and just define a mesh, but it doesn't stop it from being confusing. I define a node, and define the node type as a mesh, and then point it to a geometry and a material. I don't know how to handle this if there are multiple materials, so I might end up doing what I did before, where I cheated and combined the textures into one image.
The materials are also pretty weird, I guess the designers of the file format assumed that a scene would contain several of the same materials, and you define the materials all in one common area. Again not horrible, since it's in the node definition where I pair the geometry with a material for a draw call. But that ends up being the biggest problem with the whole file format, is you have references to all of these other parts of the file, and you have give them all names and manage what's being linked with what. Which is why I find even something link MML a lot more intuitive, where you have a list of models in a scene, and each model has it's own internal definition. So you're not having to go through and manage all of these relationships, you just read the model.
Edit:
So another quick debug and the straight convert to and from works. So I need to be careful and try and get the exact form, just broken up into pieces so I can add values into it.
Edit:
No bones (yet). But it's a lot easier working with objects than it is tying to jam a bunch of strings together. So now it's mostly a matter of being able to manage the different id's and references that Collada forces. I have two things to work on next. One is changing the material and maybe potentially the draw calls.
I'll ask on discord (and maybe twitter), and here is fine too. Would people prefer multiple individual textures or one large texture? For example Roll has three different pallet image combinations:
So I can either export them separately, or group them into one image:
I think that individual images would have multiple draw calls, though three really isn't that many. Basically it could be better for editing as I think the textures could be edited by hand. The grouped image would have a lot of wasted space for the size, but 512x512 might not be too bad. The last option might be to try to make something automated where I take the min u, max u, min v, max v from each texture and try to group it automatically onto it's own texture. Though I'm not sure how much of my sanity meter it would take to recalculate the uv's. Or I could set it as an option for which texture grouping is desired.
I think i'll try implementing the multiple texture option first, as that is more general purpose. And if there's only one texture it will work the same anyways.
Edit:
Looks like there are online sprite generators like: www.leshylabs.com/apps/sstool/. I'll try and see if there's anything in library form that I can include into my project. If not, then I'm not a big fan of having to figure out how to cut and arrange a group of images to use the most efficient amount of square space.
Edit:
Looks like there is an MIT javascript texture packer: github.com/andrewlowndes/image-packer Looks like you provide a list of width and heights, then it figures out the size of the square and how they fit into the box, and gives you the coords. So the only thing you have to do is create the canvas and draw to those coordinates. Though this will throw out any information in the face texture sheets, so it really is a matter of pick your poison.
Edit:
I can't seem to find any example models with multiple materials to figure out what syntax I need to implement multiple textures, so I think I'll end up going with the image packing option by default. For faces I think I can add in an option if the width and height of a texture is 256x256, and that will take precedent over the range of uv values. And I think that will be a good balance that will make sure that textures are square, textures only use what they need and face selection pallets will be included.
So basically assuming that there is only going to be one texture allows me to use preset values for just about everything. And then just fill in the position, normals, uv's and triangle indices. Which means I have two steps to work on next. One is packing the images into one texture and remapping the uvs, and the other is implementing the bones on the collada exporter.
Edit:
I've been more quiet on this thread while working than I would normally like. With Collada, there's not a lot that makes logical sense, it's mostly just a matter of trying to find examples, and then matching those examples. Why are there images, materials, and effects, when you just need images and materials? I don't know. Why do you need to bind a material both at the node for the geometry in the scene, and when you draw the triangles? I don't know either. Why is it that when you just have a geometry, you declare it as a node in the scene, but when you have a geometry with bones, you have to declare the bones and controller in the scene and then link to the geometry from the controller. It really doesn't make any sense, and this is coming off the back of working with a proprietary binary format from twenty years ago, that uses 10 bit vertices and 7 bit indices.
So good news is that I've managed to get the geometry exported with bones. The problem is that I only know how to link one material and one image from the examples. So I need to join the textures. I tried the Image packer example, but one thing I quickly realized is that even if I try to cut out and only use the parts of the texture that are used, when the size of the resulting texture is rounded up to the nearest power of two, it ends up being a 512x512 texture anyways. So to keep it easy, I think I'll just assume a 512 square texture for all of the meshes, and that will make calculating the uv's and drawing the combined texture stupidly easy. (Though to be honest, I still have the same mirrored texture problem from before, so i'll have to play around with those values later).
Edit:
A few quick updates.
- I merged the textures, so that the texture will always be a 512x512 image of all of the images that make up the texture, so I only have one image to export for .dae
- Added in the 'About" section with copyright, license (GPLv3), and special thanks.
- The next thing I'm going to work on implementing is the download system. The library jsZip will let me create and save zip files. So I can add in the texture, collada file and any other formats that threejs can export to, and then be able to batch download as one file.
- After that i'll be working on finishing up changes on my collada exporter.
Exports are now available on mml.dashgl.com/tools/mml2_pc_mesh/. Every time a new model is loaded, I convert the files to .dae, .png, .json, .gltf, .glb, and populate a zip file with these formats included. You can click "export" to save the zip file with everything included. For .dae the models should be textured and rigged. The texture will point to the .png image included in the zip. In the newer Collada specification, it's possible to include textures as base64 data except Blender doesn't seem to support the more recent specification updates. So I'm using the standard set back in 2012 or something like that since it seems to be the most widely supported.
.json is the format that threejs uses natively. It should be rigged and include textures inside of it. You can drag and drop into threejs.org/editor/ to view. And threejs is becoming a popular format with some applications starting to support it. The problem is there are several versions of the Threejs file, and I'd be surprised if there were support for the latest 4.5 version as the maintainers never seem to actually want to document their format. Also animations don't seem to be included in the json export. Which is pretty dumb imo, because while there's not much support for the file beyond viewing, it can be easily used with the web. So I may have to poke around to see how animations can be included in the file for completeness.
.gltf and .glb are basically the replacement for .dae collada. I think the khronos group realized that no one wanted to support their crappy dae format, and decided to make a new format based on json because of the popularity of threejs. In my opinion it still suffers from a lot of the same problems as .dae as it's a scene format with nodes, meshes, lights, cameras, parents and children, so basically everything in a pile of random data with references describing how everything works together. It's a new file format, and while a lot of programs don't have a lot of support for it, supposedly it's the new standard so it might get there eventually. .glb is the binary format of the json based .gltf.
Not to many formats outside of that, there's a mmd exporter, but it hasn't been updated in over a year and does't have support for animations (if not bones). obj doesn't even have support for bones. And more common file formats like .fbx are closed source proprietary formats, so there's not likely going to be any support for those unless with first party support. I've toyed around with writing import plugins for blender, but the api is pretty terrible. Noesis is fun to work with, but I find it's a lot easier to work with javascript were I can create tools around what I need for how the data is arranged on the disk. The code is available so if someone wanted to write a Noesis plugin, feel free to do so.
But basically 3d file formats are a complete mess. I'm going to try and improve my collada exporter to an acceptable level so that models work with blender. Though everyday I'm more and more tempted to create a model-based file format with a blender importer-exporter, and threejs importer-exporter. If support for gltf improves it might end up being redundant, but it would still probably be better to work with a simpler format than the overblown scene formats the Khonos group keeps pumping out.
Edit:
As long as I'm thinking about it, I might as well draft it out.
0x00
0x04
0x08
0x0C
0x00
DASH
1
0x10
VERT
offset
number
weights
0x20
FACE
offset
number of groups
properties
0x30
BONE
offset
number
0x40
ANIM
offset
number
0x50
MAT
offset
number
properties
0x60
TEX
offset
number
type
The first four bytes are the magic number "DASH" indicating its a dash model. The second four bytes are an unsigned int value for the revision (in case I need to make any changes later). After that the properties for the model are defined, with the general layout being, Magic number, global offset inside the file, number of said property, and last are any properties and flags need to set for the set of values.
For the vertices we have the magic number VERT. Minimum offset should be set at 0x100 to allow additional information to be added into the header. Number is pretty straight forward, the number of x,y,z vertices for the model. The last number is the number of weights per vertex (with a maximum of 4. Bone weights should follow each vertex with a unsigned short for the number of the bone index and a 4 byte float from 0.0 - 1.0 for the weight. [x float],[y float],[z float] ([x index unsigned short][x weight float], [y index unsigned short][y weight float], [z index unsigned short][z weight float], [w index unsigned short][w weight float]).
For faces, my plan is to load more than i normally would into them. Managing the vertex colors and uv in the vertex list as a buffer geometry would be faster, but I think it would cause problems in blender with having a ton of duplicated vertices everywhere. If needed we can convert to buffer geometry on import. For faces, we want to have the material declared at the top of a group. And then specifically for each indice we want [ vertex indice unsigned short ], (, [v map0 float], , [v map1 float], [face vertex color rgba 4 bytes]), So uv map0, uvmap 1 and face vertex colors are defined by bit flags in the header.
For bones, not many properties to mess around with. [bone id nsigned short], [bone parent id signed short], [bone pos x float], [bone pos y float], [bone pos z float], [bone scl x float], [bone scl y float], [bone scl z float], [bone rot x float], [bone rot y float], [bone rot z float], [bone rot w float]. So basically id (not needed as it's in order, but good to have as a reference). Bone parent id (-1 for no parent, so signed), and then bone pos x,y,z, bone scale x, y, z, and then the rotation as a quaternion.
For animations, hypothetically this should be similar to bones where there's not a lot of wiggle room. The length of the animation needs to be declared, the fps, and then the pos, scl and rotation for each bone in each key frame needs to be declared. The main issue to figure out with this segment is how to declare multiple animations while still keeping the file format simple. It would probably be easiest to declare an ANIM in the header for every animation. The same concept could be applied to face groups, materials and textures. So the only properties with one declaration should be VERT and BONE (if present), otherwise multiples can be declared.
Materials, are my week point. And it can thing think of two ways to handle materials. My first approach would be to declare a list of binary flags in the header for what properties are defined in the mesh. Though that could get ugly really fast as I'd have to really document the order in which they appear and what each of their formats is. Plus that would make adding in new properties a complete pain. So the second option I think would be easiest, declare a magic number for each property, so DIFF, SPEC, ALBD (alpha blend). Where the bytes can be read, and then know what comes after it. That way new properties can be defined and implemented if needed, and you should be able to look at the file and see what's going on.
Textures, this one should be pretty straight forward, just inclde the binary data of the image. And then the only flag on the end is for the type, so png = 0, jpeg = 1, and just have a list that gets managed. Not really a fan of including mipmaps into the file, since I think those can be generated later, but it shouldn't be too much of a problem to add those in later if needed. So it might be a good idea to leave a few bytes open so I don't have to break compatibility if needed.
So I think that's about it, this should be a simple specific file format, that doesn't provide everything and the kitchen sink, but should be fairly versatile and handle a lot of use cases for general modeling use. So taking what I've written, the file format should look something like this:
DASH
1 (revision)
0x10 (start of header)
11 (number of fields in header)
0x10
BONE
Offset
12 (number of bones)
0x20
VERT
offset
200 (number of verts)
1 (influence per vertex)
0x30
TEX
Offset
0 (tex number)
0 (png)
0x40
TEX
Offset
1 (tex number)
0 (png)
0x50
MAT
Offset
0 (mat number)
11 (num properties)
0x60
MAT
Offset
1 (mat number)
11 (num properties)
0x70
FACE
Offset
0 (mat number) / 100 (num faces)
8 (face flags)
0x80
FACE
Offset
1 (mat number) / 120 (num faces)
8 (face flags)
0x90
ANIM
Offset
0 (anim number)
18 (number of frames)
0x0a
ANIM
Offset
1 (anim number)
30 (number of frames)
0x0b
ANIM
Offset
2 (anim number)
30 (number of frames)
Hmm, kind of weird. I have references and numbers kind of jumbled together. I kind of want that to line up more cleanly, so I might try and define number of, id's and references as shorts, and leave properties on the end. Since there's not that much data to be included, shorts should be enough to work with. And then I want each block to be ANIM (length) and then be able to start parsing the data directly without having to read too many more values than that.
[Magic Number 4 bytes] [Offset 32 bit] [Id short] [ref short] [number of short][properties short]
Edit: A couple of notes.
For bones i'm thinking about what will be needed. Looking at threejs and collada it looks like they just export the elements of each bone and read them back in. So the closer I can get on these formats to other ones, the more I'll be able to reference pre-existing source code for these formats.
For vertices, it looks like everyone just assumes up to four weights. So it might be a waste of space, but I might as well keep in line with what exists and throw four in there. So the next question is do i want to set a flag for if weights exist or not, or assume weights if bones exist.
Edit:
Hmmmm, it did not take very long to get a basic exporter working for my file format: pastebin.com/hV3uHEzX. Don't have animations implemented, but I need to start working on a threejs importer first to make sure I can get back what I put in first. If I can get and exporter/importer for threejs, I can start looking into writing a plugin for blender.
Edit:
I have the basics working on an importer exporter. The exporter is working with everything except for animations. The importer supports vertices and faces. And will work on implementing bones and textures next. Preview of exporter can be seen here: pastebin.com/942w4azf
Edit:
Lmao, managed to get threejs exports with textures and rigging imported back into threejs using this file format. Right now I'm grouping all of the textures into one 5125x512 image because I can only figure out how to export with one material/texture with collada and it's causing some uv issues. Later, I'll set it back to rendering individual images, because it just works a lot easier that way. A little more testing and I should be onto animations. Mostly I need to declare fps, length, and then pos, rot and scale for each bone.
Edit:
I've got the documentation for my file format laid out here: format.dashgl.com/. And I've managed to write an exporter and loader for threejs to export models to a binary file, and re-import the model with everything working, including animations. So I'm going to try re-organizing the triangle face declaration in my file format, and fix an issue with the animation length, but aside from that. The file seems to be working pretty well.
That means two things. One that I can start working on a blender addon to load my file type. And the second means that I'll probably rewrite my mml2 pc mesh page to only use my file format. Basically i'll leave the version that's online as-is, but i'll make a copy of it, remove the image preview option, not join the textures, and see if i can do anything about the texture flip problem.
So here's the JSON string for animations of the Collada simple rigged figure. And just like everything else in Collada, you can't just define something. You have to lay out the values, create an id, create something else, reference the last id, and then add another tag and reference the last id. But on the bright side in this example, it looks like they're exporting what looks to be an inverse transform array. So the next step will be to load this figure into threejs and poke at the animations, and see if these values are present in the animation.
So the issue that I have with 3d file formats is that there is no "just right" option. On the simple end you have obj, which is good for quick testing but lacks most required features like bones, or animations. And having the materials and textures being split up into different files doesn't help.
On the middle ground you have the threejs json model format, documented here: github.com/mrdoob/three.js/wiki/JSON-Model-format-3. The Confusing part about this is that most of the sample json models in the docs use this format. Most of the support for plugins (the few that exist) use this format. And yet it's Deprecated. Version 4 is not defined, and when you use the toJSON call in the library to export a model, it doesn't include animations.
On the overly complicated end you have collada. Specifically the .dae format, where everything is a reference of a reference of a reference. And you basically declare a list of data for something. Then point to that list of data and say what that data is. And then you define another list, where you point to the last list you made, describe it again and then use this indices. And then that list is referenced from the geometry, which is referenced from the scene, which is then referenced from the list of scenes. And pretty much everything gets to the point of beyond the meta-physical and you start to wonder just what kind of god would allow a file type like this to exist.
The gltf format is a new format introduced by collada, to address the short comings of .dae. And if it gets enough support that enough programs support it, then it would be nice simply be able to export to a format with plugins and support that other people have developed for it. But from my experience, it's not a file format that I would want to interact with directly and write tools specifically for (if i can avoid it).
So enter the "Dash Model Format" (.dmf), which is defined and documented here: format.dashgl.com. It's basically a small binary format that has lists of information to describe a model. So first you have a list of bones. Then a list of vertices, then a list of textures, then a list of materials, a list of triangle faces. And finally a list of animations. So it's so stupid that even an idiot like me can use it. You have a list of two textures (0, 1). And a list of three materials (0,1,2), and three face groups (0, 1, 2). Material 0 uses texture 0. And Face 0 uses material 0.
So I have an exporter and loader implemented in threejs. I was able to export a model to the .dmf format. And then write an import script for drag and drop, and get the original model back and working with animations working again in threejs. So I might make a few more tweaks to the model format, specifically the face list at the moment. And when I'm done I'll fill in the threejs page on the format documentation. If anyone is interested in taking a look at it, the source for the format is up on gitlab: gitlab.com/kion-dgl/DashModelingFormat.
Which means the next order of business is to start working on a Blender Addon that can import the format, so that people can export .dmf from the web and then be able to import it into Blender so that they can edit the models. So this post will be my notes about implementing a Blender addon and all of the pain and anguish that goes with it.
To start I'll post the document api reference: docs.blender.org/api/. The "current" version currently references 2.79b, which is what I'll be using as a reference to try and give the addon the biggest chance of being supported without change in future versions. But for file imports I would be surprised if there's anything terribly version breaking. And will see if it works with older versions of blender, simply because the api features i'll use probably haven't changed.
And to define an import and export into the blender menu, they have: def menu_func_import(self, context): self.layout.operator(operators.fileio.ImportPmx.bl_idname, text="MikuMikuDance Model (.pmd, .pmx)") self.layout.operator(operators.fileio.ImportVmd.bl_idname, text="MikuMikuDance Motion (.vmd)") self.layout.operator(operators.fileio.ImportVpd.bl_idname, text="Vocaloid Pose Data (.vpd)")
def menu_func_export(self, context): self.layout.operator(operators.fileio.ExportPmx.bl_idname, text="MikuMikuDance Model (.pmx)") self.layout.operator(operators.fileio.ExportVmd.bl_idname, text="MikuMikuDance Motion (.vmd)") self.layout.operator(operators.fileio.ExportVpd.bl_idname, text="Vocaloid Pose Data (.vpd)")
Search for "menu_func_import" in the blender documentation and you get:
Nothing!!! Isn't blender wonderful? Which again is what I find so funny, is that you expect this from undocumented proprietary formats for the Playstation. but you don't expect this crap from open source projects. So it looks like I'll have to take what i can from python plugins I can find online to compare and look for hints.
There's also the mysterious bpy module, which you have to notice the tiny little link on the bottom right hand side of the documentation: docs.blender.org/api/current/py-modindex.html
Edit:
So we've confirmed there's no tutorials for import addons. And not even any real documentation for the functions you have to define inside the addon. So let's start with the most fundamental part of where you're supposed to put the scripts. So according to this answer it looks like we're supposed to put a folder here:
So after doing some more searching, I found that if you press Shift+F11 in Blender, it will bring up the text editor. And inside of that is a menu for Template -> Python ->Operation Import Export. And the template gives you this: pastebin.com/CHf7tb3d
And I can copy the header from other plugins and start with: