Difference between revisions of "Talk:FA1C39F7"

Latest revision as of 21:23, 3 November 2010

[edit] Say What?

I`ve been investigating this record format because it is one of the most numerous formats in a lot file that I believe is corrupted. Unfortunately, I have been having some difficulty with the record format as described here. Still, I have been able to decode nearly all of it. However, as my understanding of the record format has improved, I have discovered that the data I am seeing does not quite match the description. In the hope that someone could help me [Please?], and that my experience might help others, and not wanting to mess up the existing description with my misunderstandings, I have documented my observations here. Even though my understanding of the format has improved considerably since originaly posting this commentary, and I have updated my description to match, I am still working on this record at the time of writing.

I start with some relatively simple structures that occur a variable number of times in the record, and build my way up towards the full description:

See notes.

I call these blocks "Slot" blocks because some of the "Name" fields actually have the word "slot" as part of their value, and because the co-ordinates they contain often [if not always] make perfect sense as the object-relative locations of the object features mentioned in the correspoding "Name" field.

I had to invent a name for these blocks, but was singularly uninspired at the time. Any suggestions?

[edit] Notes

After the main block, additional data occurs that varies with the type and version of the outermost block, but which does NOT appear after a block of the same type and version within another block:

• cObject blocks:

• When the main block is a cObject block of version 15, it is [always?] followed by nothing at all, marking the end of the record.
• When the main block is a cObject block of version 16, it is [always?] followed by a single additional DWORD [always 0x00000000?].
• When the main block is a cObject block of version 17, it is [always?] followed by one of the following:

• Nothing at all. Always marks the end of the record.
• A single DWORD, always 0x00FE0000. Always marks the end of the record.
• A DWORD Count [usually 0] with Count additional DWORDs. Always marks the end of the record.
• A DWORD Count [always 2 or 3?] with Count Blend blocks [always?] describing a platter of Appetizers, Cheese, or Chips.
• When the main block is a cObject block of version 14 , the format is unknown, as is the format of the version 14 cObject blocks [and cObjectBlock blocks] themselves.

Other versions of cObject blocks have not been seen.

• cAnimatable blocks:

• When the main block is a cAnimatable block of version 14 or 15, the format is unknown.
• When the main block is a cAnimatable block of version 16, it is always followed by one of the following, all of which mark the end of the record:

• Nothing at all.
• A single DWORD, always either 0x00FE0000 or 0xFD000000.
• A DWORD Count with Count additional DWORDs. The value range on the count and content of these DWORDs is severly restricted: only five different patterns have been observed!
• A DWORD Count with Count Anim blocks.

Other versions of cAnimatable blocks have not been seen. Version 17 blocks apparently do NOT exist.

• cPerson and cLocomotable blocks.

Because of the way that cPerson and cLocomotable blocks always occur only with a single instance of the other, it is impossible to determine if any data present belongs to the cPerson block or the cLocomotable block. At any rate, the cLocomotable block`s embedded cAnimatable block has been observed to be followed by any of the following:

• A DWORD Count followed by
• Two DWORDs, always one of

• 0x00000000, 0x00000000
• 0x00000001, 0x00000000
• 0x00000001, 0x00FE0000

Always marks the end of the record.

• A single DWORD 0x1D000000 followed by some unknown data containing a 7BITSTR "Routing - Eye Attractor " plus a decimal number in parentheses, 86 bytes plus the length of the number. Always marks the end of the record.
• A single DWORD 0x1E000000 followed by some unknown data containing a 7BITSTR "Portal - Pedestrian " plus a decimal number in parentheses plus " (bbox)", 89 bytes plus the length of the number. Marks the end of the record.
• A single DWORD 0x2B000000 followed by some unknown data containing a 7BITSTR mentioning a Sim by neighborhood and Character file name, a decimal number in parentheses, and "(glasses_grip)"; 98 bytes, plus the length of the Sim name. Example string: "E001_User00137 - Ralph (158) (glasses_grip)". Always marks the end of the record.
• A DWORD Count with Count Anim blocks, optionally followed by:

• Nothing at all. Always marks the end of the record.
• Two DWORDS, always one of
• 0x00000000, 0x00000000
• 0x00000001, 0x00000000

Always mars the end of the record.

Other data may follow the Anim blocks, or appear in place of them, but this data has proven very resistant to analysis.

As part of my ongoing investigation, I have used my tool to test-read every lot in UNPLAYED neighborhoods E001, F001, G001, N001, N002, and N003, [all of the standard main neighborhoods, but none of the standard subneighborhoods] and still have some troubles with the data following the cLocomotable`s embedded cAnimatable block. As of the last scan of the file it resides in, at least one cPerson record had a Anim-block count of 3, followed by TWO Anim blocks and something that clearly is NOT an Anim block as described above. The record in question is Neighborhoods\G001\Lots\G001_Lot23.package, TypeID=0xFA1C39F7, GroupID=4294967295, InstanceID=218. Several other files have what appears to be one or more of these non-Anim blocks at the end of one or more cPerson records, but so far I have only found the one record with such within the *counted* Anim blocks. Update: I may have found additional examples. Also, it now appears that there is an alternate format of the Anim block which is shorter than the one documented above.

I still don`t know how to identify the presence of the Anim blocks by any way other than checking for the length of the unused data at the end of the record, but that method had proven absolutely reliable until that abberant record was found, and has proven reilable for nearly all other records observed to date, especially if the odd data really is an alternate format of the Anim block.

Later notes:
I had been updating this page daily [sometimes twice daily!] with the results of my latest investigations, until I fell ill and was unable to perform significant analisys, nevermind post the results. Since recovering, I have confirmed the existance of multiple Anim block lengths, but have as yet been unable to find any data whatsoever that reliably predicts the amount of data between the second and third 7BITSTR values. Because of this, I have written some code that searches the data blocks for reasonably-valid 7BITSTR values, and while I cannot yet predict where they will occur in the Anim block, it seems that they *ALWAYS* occur at one of six different places. Additionally, "Name?" and "Type?" are apparently never 0 bytes long unless they are *both* 0 bytes long, in which case the rest of the Anim block appears to be far shorter indeed, and the third 7BITSTR, if it exists for this format, is also 0 bytes.

Ugh. I have been working on this for about a MONTH now, and am heartily SICK of looking at numbers all day long for weeks at a time. Bleah! Unfortunately, it is not possible to do sigificant additional analisys on the remainder of the record until the remainder of the record can be reliably located, and it is not possible to reliably predict the location of same without knowing the length of the [CENSORED] [BLEEP] Anim data which preceeds it. [Sigh.] Even though my analisys tool now performs at least several hundred to a few thousand times as fast as before I fell ill [because I automated some data collection I had previously been performing manually], analisys is still proceeding at a far slower rate than before, due to a combination of having become analisys-weary and having already done all the easy stuff.

Latest notes:
Did El Diablo himself work on any of the EA Sims2 expansions-programming teams? The Anim blocks are anything but simple!

My automated data-collection code [named "remember()"] stores a (single) copy of every byte seen in each position in each known length of any given named field [e.g., field Anim.Unknown4 with value-sets named Anim.Unknown4.008.377 for all values observed in position 8 of all 377-byte instances of field Anim.Unknown4, for all positions in all observed lengths]. When told to save observed values, it rewrites part of the program source, completely replacing one of the source files. That source file makes for some interesting reading on occasion.

I also have a "guessLength()" function that compares that stored data to the field under examination and reports either the length that best matches the data, or the fact that no known lengths match. This function is good at guessing the length of the data block, but cannot be perfect unless all possible legal values have been seen in all locations.

I also have a "findString()" function that scans through data, searching for the first well-formed 7BITSTR that it can find. Except for a single 0x00 byte [which would be a legal 7BITSTR of length zero], this function reports the first location within the data that contains a count followed by count-many bytes whose values are acceptable as ASCII text. While this will report occasional false positives [arbitrary data that just happens to fit the definition of valid text, as defined within this function], if it reports that the first non-empty string is at position X, there is guaranteed to not be any valid non-empty string prior to position X.

Using these functions to locate and display the text values embedded in Anim blocks has resulted in finding Anim blocks where the length of Unknown1 is any of {12, 16, 20}. It has also resuled in finding Anim blocks where the length of Unknown4 is any of {26, 313, 361, 369, 373, 377, 381, 385}. At this time, I cannot guarantee that other lengths do not exist. Anim blocks with Unknown1 fields of length 20 are proving most difficult indeed to parse. All other Anim blocks seem relatively tame. --GeneralOperationsDirector