Sign in to Mod The Sims
#26
1st Oct 2019 at 9:50 PM
1st Oct 2019 at 9:50 PM
Advertisement
#27
2nd Oct 2019 at 8:21 PM
2nd Oct 2019 at 8:21 PM
Original Poster
#28
3rd Oct 2019 at 5:07 AM
3rd Oct 2019 at 5:07 AM
#29
4th Oct 2019 at 3:00 PM
4th Oct 2019 at 3:00 PM
#30
4th Oct 2019 at 4:08 PM
Last edited by Lyralei : 4th Oct 2019 at 4:28 PM.
4th Oct 2019 at 4:08 PM
Last edited by Lyralei : 4th Oct 2019 at 4:28 PM.
I feel like whenever we have the talk about "Shaders" people actually don't necessarily understand the difference between "Materials" or "Shaders"
Here's a great video on how shaders are made:
And materials, are mainly texture-based. They both need each other though to create the final results on a mesh
(See: https://forums.cgsociety.org/t/shad...e-same/927326/6)
Now, coming to your question. Shaders are, unfortunately, all engine based. Given how TS3 has been made with HLSL shaders (and I think GLSL for mac copies IIRC), you can't just go in and edit it, no tool can do it. You'll need to have the development build of TS3 itself to adjust the code of shaders. OR Do what this dude did with another DirectX9 game: https://forums.taleworlds.com/index.php?topic=296795.0
EDIT: Correction, TS3 uses OpenGL But does indeed have HLSL Shaders
But as you can see already, it requires 3 things. 1. C++ knowledge. 2. Knowing how the game engine works. 3. Being good at maths and algebra.
However, most people actually don't really know how much EA's shaders can already do by changing values in the MATD file or even the LITE file. For the LITE I always like to give people this example because even though it's a fixed tutorial, it covers so much on the main bits of, well, ts3 lighting http://poccisims3.blogspot.com/2013...-lightings.html
Here's a great video on how shaders are made:
And materials, are mainly texture-based. They both need each other though to create the final results on a mesh
(See: https://forums.cgsociety.org/t/shad...e-same/927326/6)
Now, coming to your question. Shaders are, unfortunately, all engine based. Given how TS3 has been made with HLSL shaders (and I think GLSL for mac copies IIRC), you can't just go in and edit it, no tool can do it. You'll need to have the development build of TS3 itself to adjust the code of shaders. OR Do what this dude did with another DirectX9 game: https://forums.taleworlds.com/index.php?topic=296795.0
EDIT: Correction, TS3 uses OpenGL
But does indeed have HLSL ShadersBut as you can see already, it requires 3 things. 1. C++ knowledge. 2. Knowing how the game engine works. 3. Being good at maths and algebra.
However, most people actually don't really know how much EA's shaders can already do by changing values in the MATD file or even the LITE file. For the LITE I always like to give people this example because even though it's a fixed tutorial, it covers so much on the main bits of, well, ts3 lighting
http://poccisims3.blogspot.com/2013...-lightings.html
#31
6th Oct 2019 at 11:09 PM
6th Oct 2019 at 11:09 PM
#32
7th Oct 2019 at 12:10 AM
7th Oct 2019 at 12:10 AM
#33
14th Oct 2019 at 11:41 PM
14th Oct 2019 at 11:41 PM
#34
23rd Dec 2019 at 4:54 PM
Last edited by mixa97sr : 23rd Dec 2019 at 5:12 PM.
23rd Dec 2019 at 4:54 PM
Last edited by mixa97sr : 23rd Dec 2019 at 5:12 PM.
I've opened the precomp file in a hex editor, I'm looking at it right now and... it might actually be a file with shader definitions. It is a file where shader coding is stored. From what I see it has a lot of shader data going for it and it's structured like this:
- Header with file type and probably also libraries used
- Vertex and Pixel shader data for each shader
- Value Type definition (int, float3...) for each property entry per shader, for all shaders
- Some form of parameter list but I'm not sure if it's actual parameters described previously or something else. From strings it goes like: TECH, PARM, PARM, PASS
- Final list of shaders described in the file
If this is trully the same thing as sims 4's precomp file (which I just compared and they really are 90% the same) then, it should be able to change the existing shaders and add new ones. Because few people managed to actually add new shaders and edit existing ones using the .BT file gurus provided them with.
If I am correct is, how you would do this is you would run binary remplate (.BT) file over shader precomp file and you would get a version that's readable and editable. The thing with sims 4 is that you would be able to have that decompressed file in mods folder work. For sims 3 that is most likely not the case meaning that you'd probably have to compress the file again... which I don't think it's possible.
EDIT: I read Lyra's explanation and I guess she might be right though... that would mean that people who got this to work with sims 4's precomp shader did not actually get it to work? I'm currently in the state of confusion.
I guess I'm hoping too much
- Header with file type and probably also libraries used
- Vertex and Pixel shader data for each shader
- Value Type definition (int, float3...) for each property entry per shader, for all shaders
- Some form of parameter list but I'm not sure if it's actual parameters described previously or something else. From strings it goes like: TECH, PARM, PARM, PASS
- Final list of shaders described in the file
If this is trully the same thing as sims 4's precomp file (which I just compared and they really are 90% the same) then, it should be able to change the existing shaders and add new ones. Because few people managed to actually add new shaders and edit existing ones using the .BT file gurus provided them with.
If I am correct is, how you would do this is you would run binary remplate (.BT) file over shader precomp file and you would get a version that's readable and editable. The thing with sims 4 is that you would be able to have that decompressed file in mods folder work. For sims 3 that is most likely not the case meaning that you'd probably have to compress the file again... which I don't think it's possible.
EDIT: I read Lyra's explanation and I guess she might be right though... that would mean that people who got this to work with sims 4's precomp shader did not actually get it to work? I'm currently in the state of confusion.
I guess I'm hoping too much
#35
23rd Dec 2019 at 9:52 PM
23rd Dec 2019 at 9:52 PM
Yeah so basically, the mentions (as far as I've been able to figure out) Are merely like "This shader uses use of Pixel shader data and vertex shader data, etc." Which would define, "Hey I am this shader and I am using this amount of specular, colour at this and that vertice". That is, as far as my knowledge of shaders go. See: https://www.tutorialspoint.com/dire...ixel_shader.htm
But removing the binary bits of the hex code and re-save it after the changes is actually pretty easy to program However, this info needs to be the SAME as the info from the engine.
When I referred to this, I basically meant that the actual shader settings are in the TS3W.exe and TSLHost.dll (Although the TSLHost.dll is I think just to contact EA's server with the whole sharing gifts in-game with your friends). The Shader data from there uses variables and references that are defined in the Precomp file we're talking about here. I assume this is because of saving loading time and getting easier access to adding more Variables rather than re-compiling the engine over and over again. Hence the mention of "Floats3, etc"
YOu could actually check out how they did it with Minecraft's shaderlab. Minecraft was supposed to be non-crackable regarding shaders, but I think they reversed engineered it in some ways. I haven't actually checked it out though.
But removing the binary bits of the hex code and re-save it after the changes is actually pretty easy to program
However, this info needs to be the SAME as the info from the engine.
Quote:
| So when you're compiling your code for the game, you obviously want the game to be able to call its textures, shaders, rendering settings, etc. Now I'm not sure if that's common practise perse but in the TS3W.exe as well as the TSLHost.dll, it holds pretty much all the 'directions' of where the game should get its details. They're actually listed like a map structure, which was obviously needed before the game was actually being compiled for release. |
When I referred to this, I basically meant that the actual shader settings are in the TS3W.exe and TSLHost.dll (Although the TSLHost.dll is I think just to contact EA's server with the whole sharing gifts in-game with your friends). The Shader data from there uses variables and references that are defined in the Precomp file we're talking about here. I assume this is because of saving loading time and getting easier access to adding more Variables rather than re-compiling the engine over and over again. Hence the mention of "Floats3, etc"
YOu could actually check out how they did it with Minecraft's shaderlab. Minecraft was supposed to be non-crackable regarding shaders, but I think they reversed engineered it in some ways. I haven't actually checked it out though.
#36
24th Dec 2019 at 1:25 AM
24th Dec 2019 at 1:25 AM
#37
27th Dec 2019 at 1:19 PM
27th Dec 2019 at 1:19 PM
So, I took a look into the engine with a hex editor, and it seems pretty straightforward. What I know is that the Engine is made with C++.
Some of you might recognise the mashalAs stuff whenever you're working on things. Here's an example:
Now, marshalAs is primarily used to import C++ dll libraries (IIRC). C++ is also usually the preferred language for making engines, because it's the almost closest language to computer language. TS3's engine is also custom, so it makes sense why they'd use C++.
Now, in the engine, if we were to look at the code I just copied in my post, it would loook like this:
Which makes a lot of sense! But if we were to do this with shaders, I think we'd have to make the communication work through editing hex code, so I'd figure making an application that adds the bytes to the .exe file would be better if you were to want end-users (So in our case people who download CC) to use it and safetly install such a thing.
But, of course, currently i'm talking about about functions in the engine And not shader editing :p. But I've got my reasons!
Some useful stuffz:
http://timjones.io/blog/archive/201...-cbuffers-using
https://modthesims.info/wiki.php?ti..._Modding_Basics
Personally, I think i've got myself a lead as to how to add custom materials in a sense like this: https://docs.unity3d.com/Manual/SL-...erExamples.html
But that yet needs to be seen. If I get that working, i'll make a tutorial for it!
Some of you might recognise the mashalAs stuff whenever you're working on things. Here's an example:
Code:
[DllImport("Sims3Common.dll")]
[return: MarshalAs(UnmanagedType.I1)]
public static extern bool World_AddObjectToObjectManagersImpl(ulong scriptHandle, ulong groupId, Vector3 position, int level, Vector3 createFacing, uint hiddenFlags);
Now, marshalAs is primarily used to import C++ dll libraries (IIRC). C++ is also usually the preferred language for making engines, because it's the almost closest language to computer language. TS3's engine is also custom, so it makes sense why they'd use C++.
Now, in the engine, if we were to look at the code I just copied in my post, it would loook like this:
Code:
ScriptCore.World::World_AddObjectToObjectManagersImpl
Which makes a lot of sense!
But if we were to do this with shaders, I think we'd have to make the communication work through editing hex code, so I'd figure making an application that adds the bytes to the .exe file would be better if you were to want end-users (So in our case people who download CC) to use it and safetly install such a thing.But, of course, currently i'm talking about about functions in the engine And not shader editing :p. But I've got my reasons!
Some useful stuffz:
http://timjones.io/blog/archive/201...-cbuffers-using
https://modthesims.info/wiki.php?ti..._Modding_Basics
Personally, I think i've got myself a lead as to how to add custom materials in a sense like this: https://docs.unity3d.com/Manual/SL-...erExamples.html
But that yet needs to be seen
. If I get that working, i'll make a tutorial for it!
#38
28th Dec 2019 at 5:31 AM
28th Dec 2019 at 5:31 AM
#39
31st Dec 2019 at 10:29 AM
31st Dec 2019 at 10:29 AM
So, @mixa97sr was pretty right! Except for one thing here, which is whether the ts3 precomp file is the same as ts4, and even I was fooled into thinking this, so no worries.
TS4 only has functions calling the shaders. That's it. Ts3 actually has genuine functions as to how the shader should act, be, do, etc. Hence why you'd see references like "Tech" and "Pass". So those are here for a reason Else we'd have been able to find a JSON file or something that TS4 has to be able to customise it without cracking open that file over and over again and recompile the engine every single time, given how they are loading in a list. But that's TS4, not TS3 
There are about 604 shaders in the 1.67 patched game. Each have their own "chunk" So to speak. They're string-wise, only mentioned twice. (For example, search for PreviewWallsAndFloors, and you see what I mean). Now, you of course got the FNV32 string version of the shader's name as well and that one is mentioned plenty of times if you were to search it. The first mention of say PreviewWallsAndFloors, in the SSET for the shader, happens to have some garbage bytes and a list of some sort. The end bit of the bytes chunk, however, has the naming of the values (which I presume you can customise to something else? As long as it's done in this file) It contains the default settings of a shader param, but also a "Min" and "Max" value. For textures this is obviously a bit different.
Later on, if we use the FNV32 Hash of PreviewWallsAndFloors, The FNV32 hash is mainly used as an "ID" for the "TECH" Which makes a lot of sense: https://docs.microsoft.com/en-us/wi...ques-and-passes
Just like how we saw that SSET has it's own "Shader" chunk of bytes, so do the functions, which is with a tag called "SBLK" Which stands for "Shader block". See this as a wrapper for the shader functions
Later at the bottom, you've got ever shader + shader data fields, which is probably for initialising everything.... Because, EA :p
So the file is actually simple af, it's just that there are 604 shaders which makes it super big and hard to read. But this is as far as Tash and me got to reading the file. We're still working on it though!
TS4 only has functions calling the shaders. That's it. Ts3 actually has genuine functions as to how the shader should act, be, do, etc. Hence why you'd see references like "Tech" and "Pass". So those are here for a reason
Else we'd have been able to find a JSON file or something that TS4 has to be able to customise it without cracking open that file over and over again and recompile the engine every single time, given how they are loading in a list. But that's TS4, not TS3 
There are about 604 shaders in the 1.67 patched game. Each have their own "chunk" So to speak. They're string-wise, only mentioned twice. (For example, search for PreviewWallsAndFloors, and you see what I mean). Now, you of course got the FNV32 string version of the shader's name as well and that one is mentioned plenty of times if you were to search it. The first mention of say PreviewWallsAndFloors, in the SSET for the shader, happens to have some garbage bytes and a list of some sort. The end bit of the bytes chunk, however, has the naming of the values (which I presume you can customise to something else? As long as it's done in this file) It contains the default settings of a shader param, but also a "Min" and "Max" value.
For textures this is obviously a bit different.Later on, if we use the FNV32 Hash of PreviewWallsAndFloors, The FNV32 hash is mainly used as an "ID" for the "TECH" Which makes a lot of sense: https://docs.microsoft.com/en-us/wi...ques-and-passes
Just like how we saw that SSET has it's own "Shader" chunk of bytes, so do the functions, which is with a tag called "SBLK" Which stands for "Shader block". See this as a wrapper for the shader functions
Later at the bottom, you've got ever shader + shader data fields, which is probably for initialising everything.... Because, EA :p
So the file is actually simple af, it's just that there are 604 shaders which makes it super big and hard to read. But this is as far as Tash and me got to reading the file. We're still working on it though!
#40
31st Dec 2019 at 4:32 PM
31st Dec 2019 at 4:32 PM
#41
2nd Jan 2020 at 2:10 PM
Last edited by SmugTomato : 2nd Jan 2020 at 4:29 PM.
2nd Jan 2020 at 2:10 PM
Last edited by SmugTomato : 2nd Jan 2020 at 4:29 PM.
I've extracted all the shader names with the data in front of them into a text file in case anyone wants to try and figure out what the data in front of them means.
I've reordered them from [DWORD unknown1, DWORD unknown2, DWORD stringLength, String shaderName] to [shaderName, DWORD unknown1, DWORD unknown2].
unknown2 is 0 in all entries.
All values were read in Little-Endian but were written to the file flipped to big-endian. (Thanks java)
I've reordered them from [DWORD unknown1, DWORD unknown2, DWORD stringLength, String shaderName] to [shaderName, DWORD unknown1, DWORD unknown2].
unknown2 is 0 in all entries.
All values were read in Little-Endian but were written to the file flipped to big-endian. (Thanks java)
Attached files:
 |
shadernames.txt (52.7 KB, 60 downloads) | |
| Description: All values were read in Little Endian | ||
#42
3rd Jan 2020 at 10:13 AM
3rd Jan 2020 at 10:13 AM
Ah, nice SmugTomato! :D
I take it this is from the mKeyNameTag bit right? (Aka the bit at the bottom, with alll the strings)? I noticed that has quite a mixture of "Shaders" and "Shader params", given how they're basically strings that are being used for reading purposes.
Anyways, for the programmers out here, Tash and me made a Byte-reading template (Which was actually done quite easily by using ts4's bt file. Hence why it says that at the start still :p) We've edited in such a way that you can now read it for ts3. One note though is that for TS3, the mheapsize is being used for the shaders, whereas for TS4 they are... well, ignored. So while it would still be possible to edit it by reading the offsets, just so that you guys are aware of this.
So far, i'm still trying to figure out the linkage of bits. All I need to figure out now is just what the mplatform entries do, and how the mVertexShaderParams as well as the mPixelShaderParams communicates with the game/package files (Or well, more specifically MLOD/MODLs). So you can use that as a reference.
I used 010 editor to run the template, so make sure you've got a program that can run templates (For 010 editor, see: https://www.sweetscape.com/010editor/manual/Running.htm )
I've also included my research on what I got, which is a "Follow the DiffuseMap through the Phong shader!". mPlatform stuff however isn't included yet.
I take it this is from the mKeyNameTag bit right? (Aka the bit at the bottom, with alll the strings)? I noticed that has quite a mixture of "Shaders" and "Shader params", given how they're basically strings that are being used for reading purposes.
Anyways, for the programmers out here, Tash and me made a Byte-reading template (Which was actually done quite easily by using ts4's bt file. Hence why it says that at the start still :p) We've edited in such a way that you can now read it for ts3. One note though is that for TS3, the mheapsize is being used for the shaders, whereas for TS4 they are... well, ignored. So while it would still be possible to edit it by reading the offsets, just so that you guys are aware of this.
So far, i'm still trying to figure out the linkage of bits. All I need to figure out now is just what the mplatform entries do, and how the mVertexShaderParams as well as the mPixelShaderParams communicates with the game/package files (Or well, more specifically MLOD/MODLs). So you can use that as a reference.
I used 010 editor to run the template, so make sure you've got a program that can run templates
(For 010 editor, see: https://www.sweetscape.com/010editor/manual/Running.htm )I've also included my research on what I got, which is a "Follow the DiffuseMap through the Phong shader!". mPlatform stuff however isn't included yet.
Attached files:
| |
ShaderInfo.txt (6.4 KB, 52 downloads) | |
| Description: My research on the topic, way too long for a post so here you go! | ||
 |
Sims3_Shaders_Win32_Template.precomp.zip (2.9 KB, 37 downloads) | |
| Description: Template that you can run with 010 editor or any bytes editor. | ||
#43
3rd Jan 2020 at 10:48 AM
Last edited by SmugTomato : 3rd Jan 2020 at 11:20 AM.
3rd Jan 2020 at 10:48 AM
Last edited by SmugTomato : 3rd Jan 2020 at 11:20 AM.
Looks like my suspicions might be right about the 0 bytes just being part of a 64bit int with only the lower 32bits being used looking at your txt file, those poor wasted 1.2KB 
I was wondering why my generated FNV hashes weren't matching with what I suspected were hashes, turns out my implementation is just faulty, might be because of java's lack of unsigned integers. Should have just checked them it in S3PE to begin with.
*Edit*
Looks like I missed the bold text on the wiki stating that the string should be converted to lowercase first before hashing. Now it makes sense.
I was wondering why my generated FNV hashes weren't matching with what I suspected were hashes, turns out my implementation is just faulty, might be because of java's lack of unsigned integers. Should have just checked them it in S3PE to begin with.
*Edit*
Looks like I missed the bold text on the wiki stating that the string should be converted to lowercase first before hashing. Now it makes sense.
#44
3rd Jan 2020 at 11:58 AM
3rd Jan 2020 at 11:58 AM
That's allright! Hopefully the template makes it easier to see stuff though. Personally never worked with java before, but I've heard Python can do the job pretty well too regarding doing things on byte level. But yeah when it comes to those things, you really need to be super precise, which is annoying lol!
Currently been working on a small program that can disassemble the "mByteCode" in the VSHD/PSHD section. (see template code) With D3DCDissasembleShader. Mainly because... curiousity
and seeing how far things can be edited
Currently been working on a small program that can disassemble the "mByteCode" in the VSHD/PSHD section. (see template code) With D3DCDissasembleShader. Mainly because... curiousity
and seeing how far things can be edited
#45
6th Jan 2020 at 6:22 PM
6th Jan 2020 at 6:22 PM
#46
14th Jan 2020 at 3:21 PM
14th Jan 2020 at 3:21 PM
#47
18th Jan 2020 at 8:22 AM
18th Jan 2020 at 8:22 AM
#48
25th Jan 2020 at 10:34 AM
25th Jan 2020 at 10:34 AM
Quote: Originally posted by whiteman-Dara
| So we found out that the ghost state is shaders, fairy wings, the visual effect that is with all the effects in the file is Sims3effects, but what are visual overrides, they are also in the same place as the shaders? |
Visual overrides are actually overrides done on effects
. So, a great example could be "no more confetti!". Now, some effects can be changed in colours through creating a script mod out of it. Or, you could use this tool and method: http://www.modthesims.info/showthread.php?t=450194However, if we take the confetti visual effect as an example (and this goes quite far from what you asked), this does need a mesh and shader of course. Else, the confetti will not have 1. transparency support. 2. the way the light is defined on the confetti. So, how EA fixed this was by adding a "Particle" shader. This is basically the shader for anything confetti-like or whatever "falls" as an effect
Hope this clears it a bit! I'd recommend though if you want to know more about visual effects and editing them to start a new thread. mainly because this is about cracking open the Shader pre-comp file, and not editing visual effect.
#49
25th Jan 2020 at 1:17 PM
25th Jan 2020 at 1:17 PM
#50
25th Jan 2020 at 6:37 PM
25th Jan 2020 at 6:37 PM
|
|