|
Post by mchobo on Feb 2, 2014 12:09:17 GMT -5
I've been working on inserting custom/foreign graphics into the pc version of MML. Here is a demonstration: Original custom graphic: Cropped Texture: Modified Game Texture: Texture in game:
|
|
|
Post by Chiz on Feb 2, 2014 13:02:14 GMT -5
It looks like you found the "TM" - Texture Map - component of DASH1PC's resource files. There are four section types in the resources: BD (Binary Data), CT (Colour Table), TM (Texture Map), and WB (Wave Bank). CTs are usually a series of 32-byte palettes with 16 15-bit colours (1x5b5g5r), where colour 0,0,0 (Pure Black) is your magic pink transparency value.
EDIT: BD hasn't been touched/understood at all. It's your claim to fame if you can manage it.
|
|
|
Post by Kyle on Feb 2, 2014 18:33:12 GMT -5
Oh jeez! Don't stare at the OCG for too long, guys. It's like opening the Ark of the Covenant. Theoretically, would it be possible to make a customized texture of Teisel's face for those water ripples?
|
|
|
Post by mchobo on Feb 2, 2014 21:39:19 GMT -5
It looks like you found the "TM" - Texture Map - component of DASH1PC's resource files. There are four section types in the resources: BD (Binary Data), CT (Colour Table), TM (Texture Map), and WB (Wave Bank). CTs are usually a series of 32-byte palettes with 16 15-bit colours (1x5b5g5r), where colour 0,0,0 (Pure Black) is your magic pink transparency value. EDIT: BD hasn't been touched/understood at all. It's your claim to fame if you can manage it. Is CT a color table? A table for rendering models or something of the sort? (I'm unsure) As far as textures go, there seems to be a color table(palette) at the beginning of the TM texture file. The SUB_WPN.BIN file for example, supports my claim. It has a TM texture file, and a BD file, (Only two files) but is still able to yield texture with color. (Note: It yields color without a CT file, therefore I still do not know what the CT tables are for.) You seem to know quite a bit about the CT files, do you mind pointing me to a wiki? (I did some searching. Are CT files the same as Sony's CLUT? If so, then it make a bit more sense.) Proof:
Oh jeez! Don't stare at the OCG for too long, guys. It's like opening the Ark of the Covenant. Theoretically, would it be possible to make a customized texture of Teisel's face for those water ripples? Yes, it's theoretically possible. However, there is a limited amount of pixel space to work with. Therefore, if you desired to use his face texture, you'd need to shrink/crop/stretch/re-size his texture accordingly.
|
|
|
Post by Chiz on Feb 3, 2014 3:04:13 GMT -5
I don't have the internet right now, so my research wiki isn't publicly available, but I do have page renders of the current state of the pages. Included is the small write up for the DASH1PC BIN container header, the CT/TM formats, the DASH2PC x3/x4/x5 formats (identical to CT/TM, but more notes are on this page), and the WBK container format. imgur.com/a/4gbxx#0EDIT: Specifically, CTs are colour tables, but TMs also have a brief CT at the beginning of them, too; as such, a CT is DASH1PC's equivalent of an x3, and a TM is the equivalent of an x4. There may not be a pure only-graphics section like there is in DASH2PC (x5).
|
|
|
Post by mchobo on Feb 4, 2014 21:27:10 GMT -5
I appreciate the wiki info (Chiz) After doing further research on the structure of the TM file, I've come to the understanding that they are PSX TIM files, you may have already known that. For anyone else who may read this and may be interested in a further understanding of the structure of a TIM/TM file, these links may be useful: rewiki.regengedanken.de/wiki/.TIMwww.romhacking.net/documents/timgfx.txtAnd this may be a useful too (TIM/TM viewer)l: www.mediafire.com/download/l9agoxvoiy4awxr/PSXTimViewer+v0.1b.zip
Oh jeez! Don't stare at the OCG for too long, guys. It's like opening the Ark of the Covenant. Theoretically, would it be possible to make a customized texture of Teisel's face for those water ripples? Here is what you 'theoretically' spoke about: It's not the best crop, but it works. It has the correct palette/CLUT too (It was hexed in.) There seems to be some kind of transparent layer/texture hovering above my Teisel texture - that's why it looks off color.
|
|
|
Post by Kyle on Feb 5, 2014 8:35:09 GMT -5
We can rebuild these textures! We have the technology!
...But I don't want to spend a lot of money.
|
|
|
Post by mchobo on Feb 5, 2014 10:28:13 GMT -5
|
|
|
Post by Chiz on Feb 5, 2014 21:34:04 GMT -5
Oh, I see it now. It took me a few moments, but I see where the documentation overlaps: Len. | DASH2PC x04 | TIM format | 4B | Not present | 0x00 - TIM ID Tag | 4B | Not present | 0x04 - TIM Type Tag (BPP) | 4B | 0x00 - Palette Data + 12 | 0x08 - CLUT Data + 12 | 2B | 0x04 - Storage Address Horiz. | Palette Org. X | 2B | 0x06 - Storage Address Vert. | Palette Org. Y | 2B | 0x08 - "Width" (usually 16) | Colours per CLUT (16 in 4BPP and 256 in 8BPP) | 2B | 0x10 - "Height" (usually #Palettes) | Number of CLUTs | x | - - PALETTE DATA | - - CLUT Data Block |
etc. It's odd, though, but DASH2PC completely forgets about TIM Type...which is really important, and one of the biggest setbacks I've had with having a bulletproof texture extractor for it; I can't tell if the image is 4BPP or 8BPP except from educated guesses based on common patterns that occur more frequently in 4BPP which are absent in 8BPP, etc. Fortunately I've only seen maybe a half dozen 8BPP images in DASH2PC, so it's been easier to just maintain an official "exceptions" list, but that won't work if modding ever becomes prevalent.
|
|
|
Post by mchobo on Feb 5, 2014 22:58:45 GMT -5
I haven't got my hands on a PC version of MML2, but if ever I do - I'll be sure to explore what I can.
Funny thing, using that TIM viewer from the link above, I was able to scan for TIMs in the PC version of MML, however I was unable to scan for them in the PSX version. Taking a closer look at both versions, they both have the same data (in terms of TIMs), but the header of the PSX version seems to be disorganization.
Moreover, the scanner turned up 0 results for the PSX version of the game, (although the data was obviously in the files.) while it turned up over 1000 results for the PC version.
Perhaps if you were to use the TIM viewer you could scan/extract the texture data from the DASH2PC version?
On a side topic, I posted some URLs above, but one of them seems to be broken. (It was working just the other day.) I'm assuming romhacking.net didn't like me using that page. Moreover, I saved that page before it was taken down, therefore if you need more info on the TIM structure, I have the info. (BTW, that page was a bit more detailed than the other.) EDIT: URL seems to be working again.
I have a question Chiz: You seem to know quite a bit about modifying these games. From what I've come to understand, there is much potential for a tool to be developed that could be used to re-texture at least MMLPC. Does such a tool not exist? I can't help but feel that all the work I am doing, modifying this games textures, has already been done by somebody else. However, I have not come across any re-textureing tools, real demonstrations, modders. Is there a community of modders that specifically modify these games? If there is, could you point me in that direction?
|
|
|
Post by Dashe on Feb 5, 2014 23:15:55 GMT -5
The orange armor's a surprisingly good look for ole' Blue Boy!
|
|
|
Post by mchobo on Feb 5, 2014 23:27:33 GMT -5
I'm thinking about making a small tool that one could use to extract the CLUT data with ease. If I do, I'll post up the source code.
Using that tool (and maybe a few others) you could change his armor palette to any other you desire.
What would be really useful though, is a tool that allows viewing of the palette beforehand.
Another thing I'd like to note, I don't see any reason why this wouldn't work on the PSX version of the game.
|
|
|
Post by Chiz on Feb 6, 2014 0:05:14 GMT -5
At least in DASH2, with the help of satoh, I was able to determine that there is compression on the resources of the game (and yes, the organization is a bit different), but once decompressed they are 1:1 identical. It is likely the same in DASH1. While the compression wasn't entirely cracked, comparing a decompressed version with the original compressed version seems to fit the definition of a 'sliding window' compression method, where the compressed data is constantly pointing backwards to copy forward bytes from the on-going decompressed data. The reason that the compression wasn't cracked was because I was having trouble identifying these pointers from normal data (without the decompressed version on hand to see and use them, entirely missing the point of compression). Incidentally, since I've proven that decompressed PS1 resources are identical to normal PC resources (within reason)...and that elsewhere, I've shown that PS1 resources can be in-place dropped into the PSP ports and have it just work...it stands to reason that the PSP ports' resources are compressed the same way, could be extracted the same way, and one could probably build a significant number of the PC version's resources without having to find and download it. It might be an interesting legal way to release a fangame or engine clone that uses those resources - have a tool that looks for ANY of the releases of the game, since they're all effectively the same.
As for there being an easy-retexture tool, no, there does not exist one. I'm not a very adept programmer/coder, only writing command line scripts and utility for automated processing (mostly isolation/extraction) of resources for further study.* Any replacements or injections I've had to do (FOR SCIENCE!) have been done manually with a hexeditor and a lot of patience. A. LOT. OF. PATIENCE. That being said, I make all my research freely available, as long as you give credit and don't profit off of it. I don't see the long-term benefit of having a texture/palette replacement tool, but I welcome them to be written if only to show that we are alive and there is interest in working on this. EDIT: * - I'm supposed to be learning COLLADA so I can output whole models with proper skeletal structure and animation. As of now, I'm just outputting OBJs of individual pieces and not even bothering with the skeleton, as I didn't understand it at the time of writing that tool. EDIT 2: Perhaps if you were to use the TIM viewer you could scan/extract the texture data from the DASH2PC version? I suppose I could, but since I already have a purpose-built utility for extracting palettes and textures (and converting them to .PAL/.ACO and .BMP while I'm at it!) I'm not sure why I'd want to use one at this point. Is there a community of modders that specifically modify these games? If there is, could you point me in that direction? The whole of cutting edge MML and MML2 modding is just me and satoh. Sorry to disappoint.
|
|
|
Post by mchobo on Feb 6, 2014 22:37:08 GMT -5
Heh, this thread is getting kinda bloated. Maybe we should break it up? Into multiple threads? Perhaps if you were to use the TIM viewer you could scan/extract the texture data from the DASH2PC version? I suppose I could, but since I already have a purpose-built utility for extracting palettes and textures (and converting them to .PAL/.ACO and .BMP while I'm at it!) I'm not sure why I'd want to use one at this point. Just a suggestion - with extracted TIMs (If any were even found), perhaps you could examine the extracted files architecture and compare it with that of the archived data.
Is there a community of modders that specifically modify these games? If there is, could you point me in that direction? The whole of cutting edge MML and MML2 modding is just me and satoh. Sorry to disappoint. Well maybe the problem is that there is no centralized community. I've found tools from other developers and videos or even demonstrations too, but there is no central spot where they all come together. Maybe Legends station could be that centralized spot. I'm not sure how anyone may feel about it, but if there was a "Game Mod" section in this forum, that'd be very useful - since I could post topics there without them being covered up by other threads that are unrelated. I guess while I'm on the topic, I'll officially make the suggestion: EDIT: mmls.proboards.com/thread/5650/game-modders-section-forumI'm lucky that I came across you Chiz. If it wasn't that I stumbled across this forum and you replied, I may have never found any other person interested in modifying this series.
That being said, I make all my research freely available, as long as you give credit and don't profit off of it. I don't see the long-term benefit of having a texture/palette replacement tool, but I welcome them to be written if only to show that we are alive and there is interest in working on this. I agree, there may be not be a major long-term benefit of a texture/palette replacement tool. But maybe anyone who ever wondered, or anyone who wanted to try some kind of modification would become interested and it'd peak their curiosity - and in turn, more newcomers would join the scene. Also, just as you said - it would show that there is life in the scene.
Now that you mention it, maybe I should get my hands on a copy of the psp version. I'm not exactly sure how far along you are in the understanding of the game architecture, but I'd love to see what you've done or where you're at. And of course, if ever I use anything of yours - credit goes where credit is due.
Moreover, I donate this code to LegendsStation(Assuming this forum belongs to the same entity): /************************************************************************************ * Title : MML CLUT Ripper (Name is subject to change) * * Author : mchobo/hobomchobo * * Email : hobomchobo@yahoo.com * * Version : 0.02b * * Date : 02/07/2014 * * Description : Currently, all this application really does is rip the CLUT from * * MML TM files (also known as PSX TIM files). Eventually, I hope to * * have it do much more than just extract CLUTs - maybe extract actual * * image data, replace the data in files, convert formats etc. We'll * * see what the future holds. * ************************************************************************************ * NOTE : This code (Although maintained by me, and may continued to be * * maintained by me) has been donated to * * http://www.legends-station.com/ for the advancement and discovery * * of game modifications in the MegaMan Legends Series. If ever I * * disappear from the scene, LegendsStation is the immediate owner of * * my code. * ************************************************************************************ * NOTE2 :I realize my code is in major need of optimization, I hope to clean* * it up as time goes on - It is something I just put together. * ************************************************************************************ * NOTE3 :If you desire to use my code, and modify it - you may do so as long* * as you share your modification with the community. Moreover, please * * give credit where credit is due. * ************************************************************************************ * CREDIT O Klarth - I used his documentation on the TIM file format to * * further understand its architecture. (stevemonaco@hotmail.com) * * O Chiz - Sharing his personal wiki, I was able to realize the * * correlation between the pc version of CT files, and the CLUT palete * * in TM files. (http://mmls.proboards.com/user/44) * * O Mirex - With his software(Unmass & Biturn) my research on TM/CT * * filetypes with made easy. (http://mirex.mypage.sk/) * ************************************************************************************ * Version -v0.01 (02/06/2014) - Inital release. * * History -v0.02 (02/07/2014) - Code Cleaned up. * * -V0.02b(02/07/2014) - Further cleaned up code. * * -v0.03 (??/??/????) --Further clean up code? Add console menu? Add * * an option for selecting a custom save/load location? * ************************************************************************************/ #include <iostream> #include <fstream> #include <Windows.h> using namespace std;
/************************************************************ *--------------Struct & Struct member functions------------* ************************************************************/
/******************************************************************** * TmHeader - Structure * *This structure takes up 8bytes of memory. It holds data for the TM* *file header of TM files. * ********************************************************************/ struct TmHeader{ WORD fileType; WORD data1; DWORD data2; };
/******************************************************************** * TimHeader - Structure * *This structure takes up 20bytes (0x14) of memory. It holds data * *for the TIM file header of TM and TIM files. If this Header is * *apart of a TM file, then its offset is 8Bytes (0x08). If the * *header is apart of a regular TIM file, then its offset is * *0Bytes (0x0). * ********************************************************************/ struct TimHeader{ DWORD Magic; DWORD Type; DWORD OffsetFromHead; //Offset from TIM Header to WORD PaletteOrgX; //Image Data (After ImageDataHeader) WORD PaletteOrgY; WORD NumColorPerPalette; WORD NumPalettes; };
/******************************************************************** * TimImageHeader - Structure * *This structure takes up 12bytes (0xC) of memory. It holds data for* *the image/texture header that resides after the CLUT of TM and TIM* *files. If this Header is apart of a TM file, then its offset * *equates to: [TmHeader Size + TimHeader Size + CLUT block Size]. If* *the header is apart of a regular TIM file, then its offset is * *equates to: [TimHeader Size + CLUT Block Size]. * ********************************************************************/ struct TimImageHeader{ DWORD ImgBlockSize; //Size includes the header and Image data WORD ImgOrgX; WORD ImgOrgY; WORD Width; WORD Height; };
/******************************************************************** * TimFile - Structure * *This structure holds the TimHeader, CLUT(Palette), and TimImage * *Header. It's the standard structure used for TIM files. * ******************************************************************** *CLUT - CLUT size is variable, however it can be detirmined by * *using one of the following equations: * *1.Size = [TimHeader.NumColorPerPalette * TimHeader.NumPaletts * 2]* *2.Size = [TimHeader.OffsetFromHead - 12] * ********************************************************************/ struct TimFile{ TimHeader PrimaryHeader; BYTE* CLUT; TimImageHeader SecondaryHeader; };
/******************************************************************** * TmFile - Structure * *This structure holds the TmHeader as well as the TimFile * *Structure. More informatin about the TmFile structure is written * *above. * ********************************************************************/ struct TmFile{ TmHeader PreHeader; TimFile FileData; };
/************************************************************ *---------------------Function Prototypes------------------* ************************************************************/
//ifstream type ifstream OPEN_IN_FILE(); //Open input file.
//ofsteram type ofstream OPEN_OUT_FILE(); //Open output file.
//streamoff type streamoff GET_FILESIZE(ifstream &); //Get size of input file.
//voide type void REPOSITION_FILE(ifstream &, int ); //Used to reposition files stream position. void DATA_TO_ARRAY_COPY(streamoff, ifstream &, BYTE []); //Copy data from file to data array. void BITWISE_ASSIGNMENT(WORD &, BYTE[], int &); //Bitwise assignment for WORD datatype. void BITWISE_ASSIGNMENT(DWORD &, BYTE[], int &); //Bitwise assignment for DWORD datatype.
/************************************************************ *------------------Start of the program--------------------* ************************************************************/
/******************************************************************** * main - Function * *The program starts here. * ********************************************************************/ int main() { //Vairable/Object Declaration. ifstream inFile; ofstream outFile; streamoff FileSize; BYTE* fileDataArray; TmFile tmDoc; int ByteCount = 0;
//Open input/output files. inFile = OPEN_IN_FILE(); outFile = OPEN_OUT_FILE();
//Get Filesize FileSize = GET_FILESIZE(inFile);
//Dynamicly allocate memory for fileDataArray. fileDataArray = new BYTE[FileSize];
//Reposition file position to begining. REPOSITION_FILE(inFile, 0);
//Read data into fileDataArray. DATA_TO_ARRAY_COPY(FileSize, inFile, fileDataArray);
//Close inFile after use inFile.close();
//Fill in tmDoc TmHeader. BITWISE_ASSIGNMENT(tmDoc.PreHeader.fileType, fileDataArray, ByteCount); BITWISE_ASSIGNMENT(tmDoc.PreHeader.data1, fileDataArray, ByteCount); BITWISE_ASSIGNMENT(tmDoc.PreHeader.data2, fileDataArray, ByteCount);
//Fill in tmDoc TimHeader. BITWISE_ASSIGNMENT(tmDoc.FileData.PrimaryHeader.Magic, fileDataArray, ByteCount); BITWISE_ASSIGNMENT(tmDoc.FileData.PrimaryHeader.Type, fileDataArray, ByteCount); BITWISE_ASSIGNMENT(tmDoc.FileData.PrimaryHeader.OffsetFromHead, fileDataArray, ByteCount); BITWISE_ASSIGNMENT(tmDoc.FileData.PrimaryHeader.PaletteOrgX, fileDataArray, ByteCount); BITWISE_ASSIGNMENT(tmDoc.FileData.PrimaryHeader.PaletteOrgY, fileDataArray, ByteCount); BITWISE_ASSIGNMENT(tmDoc.FileData.PrimaryHeader.NumColorPerPalette, fileDataArray, ByteCount); BITWISE_ASSIGNMENT(tmDoc.FileData.PrimaryHeader.NumPalettes, fileDataArray, ByteCount);
//Dynamically allocate memory for tmDoc's CLUT. tmDoc.FileData.CLUT = new BYTE[tmDoc.FileData.PrimaryHeader.OffsetFromHead - 12];
//Fill in tmDoc's CLUT. for(int countA = 0; countA < tmDoc.FileData.PrimaryHeader.OffsetFromHead - 12; ++countA) tmDoc.FileData.CLUT[countA] = fileDataArray[sizeof(TimHeader) + sizeof(TmHeader) + countA];
//Output CLUT to file. for(int countA = 0; countA < int(tmDoc.FileData.PrimaryHeader.OffsetFromHead) - 12; ++countA) outFile << tmDoc.FileData.CLUT[countA];
//Close outFile. outFile.close();
//Release dynamicly allocated memory. delete fileDataArray; delete tmDoc.FileData.CLUT;
//Print finish message cout << "Process should be complete, press enter to continue" << endl; cin.get(); return 0; }
/************************************************************ *------------------Function declarations-------------------* ************************************************************/
/******************************************************************** * OPEN_IN_FILE -Function * *Function of type ifstream. It is used to open the TM input file. * *This function takes no arguments. * ********************************************************************/ ifstream OPEN_IN_FILE(){ string FileName; ifstream File;
FileName = "C:\\Program Files (x86)\\Capcom\\Megaman\\DAT\\sub_wpn\\000-file.TM";
File.open(FileName); return File; }
/******************************************************************** * OPEN_OUT_FILE -Function * *Function of type ofstream. It is used to open the TM output file. * *This function takes no arguments. * ********************************************************************/ ofstream OPEN_OUT_FILE(){ string FileName; ofstream File;
FileName = "C:\\Program Files (x86)\\Capcom\\Megaman\\DAT\\sub_wpn\\000-file.txt";
File.open(FileName); return File; }
/******************************************************************** * GET_FILESIZE -Function * *Function of type streamoff. It is used to get the size of the * *input file in bytes. It takes one argument of type ifstream, and * *is passed by reference. * ********************************************************************/ streamoff GET_FILESIZE(ifstream &File){ File.seekg(0, ios::end); return File.tellg(); }
/******************************************************************** * REPOSITION_FILE -Function * *Function of type void. It is used to reposition the current * *location of the file stream. It takes two arguments; the first is* *of type ifstream and is passed by reference; the second is of type * *int. * ********************************************************************/ void REPOSITION_FILE(ifstream &File, int position){ File.seekg(position); }
/******************************************************************** * DATA_TO_ARRAY_COPY -Function * *Function of type void. It is used to copy data from the file to * *the data array. It takes three arguments; the first is of type * *streamoff; the second is of type ifstream, and is passed by * *refrence; the third is of type BYTE. * ******************************************************************** *It works by using a for loop, and iterates the amount of times * *equal to FileSize. On each iteration, the indicated element of * *DataArray is set equal to the byte currently in the file stream. * ********************************************************************/ void DATA_TO_ARRAY_COPY(streamoff FileSize, ifstream &File, BYTE DataArray[]){ for(int countA = 0; countA < FileSize; ++countA){ DataArray[countA] = File.get(); } }
/******************************************************************** * BITWISE_ASSIGNMENT -Function(for WORD data type) * *Function of type void. This is an overloaded function. It is used * *to combine spcific bytes from the file data array into 2 byte * *WORDs. It takes three arguments; the first is of type WORD and is* *passed by refrence; the second is of type BYTE; the thrid is of * *type int, and is passed by reference. * ******************************************************************** *It works by using bitwise operations on two different bytes. * ********************************************************************/ void BITWISE_ASSIGNMENT(WORD &A, BYTE B[], int &X){ int x1 = X++, x2 = X++; A = B[x1] | (B[x2] << 8); }
/******************************************************************** * BITWISE_ASSIGNMENT -Function(for DWORD data type) * *Function of type void. This is an overloaded function. It is used * *to combine spcific bytes from the file data array into 2 byte * *WORDs. It takes three arguments; the first is of type WORD and is* *passed by refrence; the second is of type BYTE; the thrid is of * *type int, and is passed by reference. * ******************************************************************** *It works by using bitwise operations on four different bytes. * ********************************************************************/ void BITWISE_ASSIGNMENT(DWORD &A, BYTE B[], int &X){ int x1 = X++, x2 = X++, x3 = X++, x4 = X++; A = B[x1] | (B[x2] <<8) | (B[x3] << 16) | (B[x4] << 24); }
NOTICE: This code is written in C++ and needs to be compiled. NOTICE 2: You may have to change the outFile path if your directory is protected with write privileges.
|
|
|
Post by satoh on Feb 11, 2014 9:26:48 GMT -5
Oh! Well that's new! Someone interested in modding AND capable of doing things! (No offense to everyone else)
While I can't promise I'll be back at it in the next few days or weeks (I jump around a lot) I will eventually come back to MML modding and extraction, so its nice to see that there's more interest than just myself and Chiz.
I myself recently did a bit of file injection and replacement on some disk images, though admittedly mine was more of a complicated system of hex editing. (Its not easy to load a full iso in a hex editor and replace several megabytes of it at once... a generic file injector would be a godsend for all kinds of modding... part of the problem is getting around the fact that a lot of disks require files to be at a specific offset, rather than simply having a specific name, so unpacking and repacking isn't always possible... but anyway...)
I'm assuming your code is MML1 PSX specific?
|
|