Notes for implementing LDraw Part Spec
» > >
» Goto page Previous  1, 2, 3, 4  Next
» View previous topic :: View next topic  
Here are the preliminary fields of the data structure that will be used to store LDraw models in engine.


Code:

//Random info
filename (string)
author (string)
bfcCertified (boolean)

parent (ptr, possibly null, to another ldraw object)

//Cached vertex buffers for this file and all subfiles
lineRenderCache (F32 array)
triRenderCache (F32 array)
quadRenderCache (F32 array)

//not quite sure how this will be implemented, but it will store the vertex buffer responsible for drawing the stud textures in place of stud primitives
studRenderCache (F32 array)


lineTransforms  // Point3F array of all line commands stored in *this* file
triTransforms // Point3F array of all tri commands stored in *this* file
quadTransforms // Point3F array of all quad commands stored in *this* file
subFileReferences //string array referencing the names of any .mpd/.ldr/.dat subfiles
subFileTransforms //MatrixF array controlling the transform of the equivalently indexed subfile.


and I'd like to make this a subclass of TSShape to make it really easy to replace existing models with LDraw models, and that should take care of holding the node representations (though I'm not yet sure how to handle the possibility of nodes nested within subfiles.....)

There will also be several global variables controlling stud rendering options and possibly other primitive/part-interceptions, and client connections will have "a selection depth" variable to control how recursively deep to select submodels within a model.

Finally, we'll have some sort of part cache, maintained separately from the models, to prevent commonly used pieces from having to be reparsed over and over.
<- Please Support on Lego Ideas
Dug for parts cache. In all seriousness, though, very nicely done, I of course cannot wait for the final product. Smile
Geopipe
you'll notice that there are actually several different caches Smile

one for raw parts data being read from disk and parsed into memory, and one (three actually) for storing vertex buffers for top level rendered objects, to prevent having to recurse down the entire part tree every time the model is rendered, and to allow the entire model, no matter how many poly's, to be rendered in a fixed number of OpenGL calls.
<- Please Support on Lego Ideas
Thus far today I have fixed FreeBuild so it asks the OS for the proper directory to write log files too. Real handling of this has only been implemented on OS X so far, but the skeleton is there for the others, and just needs me to actually fill in the code. This mean console.log is now written to ~/Library/Logs/Freebuild/net.cemetech.freebuild.log


I've also begun adding the ability to find + use the system LDraw repository before I bother doing any more work with lex's buffer handling.
<- Please Support on Lego Ideas
Triple posting spree ftw.




I should mention that under Windows this comes with the caveat of only being able to select LDraw directories on the same drive that FreeBuild was launched from.
<- Please Support on Lego Ideas
Because you don't know how to use windows...
DShiznit wrote:
Because you don't know how to use windows...


Actually, because I'm using the Torque file browser, and unlike intelligent operating systems, Windows treats / as the root of the current drive, not the root of the whole file system, and I haven't spent any time playing in Windows to get Torque to list drives.
<- Please Support on Lego Ideas
Well then that's torque's fault now isn't it? That being said, I have no problem with this restriction. The LDraw Library isn't THAT big, I can have it on multiple drives if I want FreeBuild on multiple drives.
DShiznit wrote:
Well then that's torque's fault now isn't it? That being said, I have no problem with this restriction. The LDraw Library isn't THAT big, I can have it on multiple drives if I want FreeBuild on multiple drives.


I'm sure I could figure it out, but I'm also having trouble thinking of a situation where someone would be really pissed off by this, so fixing it isn't high on my list of priorities. The sweet thing is that if I run FreeBuild from my mac partition, while in Windows, where my LDraw folder is also saved, I don't even have to change the path in the preferences.
<- Please Support on Lego Ideas
Reading up on the Torque Networking Library, and in particular, the fine details of Ghosting, Tickability, and NetEvents.
<- Please Support on Lego Ideas
So as far as I can tell, the lexer and parser for loading LDraw models are ready to go, and I just need to set them up to actually DO something with the data they read.
<- Please Support on Lego Ideas
For you-all's edificaton:
http://sfgp.cemetech.net/freebuild/SketchedOutline-LDrawComponents.pdf
<- Please Support on Lego Ideas
Double Post, this log shows off a couple clear issues with the parser as it stands right now.

My file loading is having difficulty locating stickered bricks in the s/ subdirectory of parts when they are specified by the full path rather than just the filename.

Secondly, the ~4KB car.dat takes at least a minute to load, whereas the ~300KB blockade_runner.mpd loads almost instantaneously. Presumably this is because the mpd loader attempts to cache things and the normal loader does not. If I then try to load the car a second time, after blockade_runner has loaded, it also loads very quickly, but I get a large number of syntax errors. Presumably this means that once the file is cached, the non-MPD loading mode can find the files there, but is having trouble actually loading them (either due to improper caching, or improper cache-loading).

Quote:
==>ldrawDebugCar();
/Users/thomas/LDRAW
Initializing LDraw Subsystem...
/Users/thomas/LDRAW
Including from disk: 4315.dat
Including from disk: stud3.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4disc.dat
Including from disk: 4-4cyli.dat
Including from disk: stud3.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4disc.dat
Including from disk: 4-4cyli.dat
Including from disk: stud3.dat
...
Including from disk: 4-4cyli.dat
Including from disk: 4-4disc.dat
Including from disk: 179.dat
LDraw Write: Part 179 moved to 4073
Including from disk: 4073.dat
Including from disk: stud4.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4cyli.dat
Including from disk: 4-4cyli.dat
Including from disk: ring3.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4disc.dat
Including from disk: 4-4cyli.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4disc.dat
Including from disk: stud.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4cyli.dat
Including from disk: 4-4disc.dat
Including from disk: 3023.dat
Including from disk: stud3.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4disc.dat
Including from disk: 4-4cyli.dat
Including from disk: box5.dat
Including from disk: box5.dat
Including from disk: stud.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4cyli.dat
Including from disk: 4-4disc.dat
Including from disk: stud.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4cyli.dat
Including from disk: 4-4disc.dat
Including from disk: 3023.dat
Including from disk: stud3.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4disc.dat
Including from disk: 4-4cyli.dat
Including from disk: box5.dat
Including from disk: box5.dat
Including from disk: stud.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4cyli.dat
Including from disk: 4-4disc.dat
Including from disk: stud.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4edge.dat
Including from disk: 4-4cyli.dat
Including from disk: 4-4disc.dat
Including from disk: 141.dat
LDraw Write: Part 141 moved to 3822
Including from disk: 3822.dat
Including from disk: s\3822s01.dat
File 's\3822s01.dat' not found in LDraw directories (We check 1 level of primitives, 9 levels of parts, 81 levels of models)
Unable to include file s\3822s01.dat, because it could not be found
Cannot find s\3822s01.dat, ignoring.
...
==>ldrawDebugBR();
/Users/thomas/LDRAW
Initializing LDraw Subsystem...
/Users/thomas/LDRAW
Cached lego10019a3s2r.ldr, used 118 tokens
caching MPD file lego10019a3s3.ldr

Caching MPD subfile lego10019a3s3.ldr
Cached lego10019a3s3.ldr, used 102 tokens
caching MPD file lego10019b.ldr

Caching MPD subfile lego10019b.ldr
Cached lego10019b.ldr, used 5505 tokens
...
==>ldrawDebugCar();
/Users/thomas/LDRAW
Initializing LDraw Subsystem...
/Users/thomas/LDRAW
Including from cache: 4315.dat
Syntax error on line "syntax error"
Syntax error on line "syntax error"
Syntax error on line "syntax error"
Including from cache: 4600.dat
Syntax error on line "syntax error"
Syntax error on line "syntax error"
Syntax error on line "syntax error"
Including from cache: 3031.dat
Syntax error on line "syntax error"
Syntax error on line "syntax error"
Syntax error on line "syntax error"
...


[edit]
Aha, diffing our parsing against the ldlite source shows that I commented out some stuff related to ftype when I rewrote the file finding. The code is set to only cache files that are are of ftype P or PART, not MODEL or OTHER. I think this is probably a big part of it. I've probably mentioned this before, but SourceGear's DiffMerge graphical diff/merge util == <3

[edit 2]
Fixed that, and and a reversed dStricmp (forgot my ! to test for equality rather than inequality). I'm not 100% MPDs are working correctly though, since I'm not getting the profiling dump at the end telling me the cache statistics, but I am for the non-mpd files.
<- Please Support on Lego Ideas
I've implemented the radix trees that will be an integral part of the in-memory cache. Before I continue with that, I need to rework the way the lexer handles an mpd document, because as of right now it caches the whole token stream without executing any of the commands within the file. Non-mpd files appear to be working fine.

[edit]
hextuple post ++
<- Please Support on Lego Ideas
CCCOMBO BREAKER

Anyway, very cool stuff elf. I can't wait to see some in-game renders that look decent Smile
CyberPrime wrote:
CCCOMBO BREAKER

Anyway, very cool stuff elf. I can't wait to see some in-game renders that look decent Smile


Me either! =) I just observed some nice features of the mpd language extensions for LDraw that make it easier to fix the current issues I'm having with the parser.
<- Please Support on Lego Ideas
Since LDraw bricks are not textured in the usual sense of needing images I should be able to do some extreme texture atlasing and cram up to 65k different 4*4 color patches into a single 1024*1024 texture. This should be more than sufficient as the total number of colors that are allowed (by the language specification) at any one time is 512. However, since each file can have its own color definitions that do not scope outside that file, it is important to leave room for models to define their own colors without overriding the default definitions. I don't expect most models to define more than a handful of custom colors, but with this scheme we could have 127 models define the maximum number of colors, and leave the default color table unpolluted.

I will need a few shaders to handle the different material properties, but this number will still be very low.
<- Please Support on Lego Ideas
Woo, and with s3tc compression you can make it so that the color map texture doesn't need much vram to use and from what I hear it is rather easy to map the 4*4 patches to the face they need to be on.

A thought I just had for the studs would be to maybe use a tiled texture of it with an alpha layer on the top of the bricks so that you only need the color map texture and the stud texture to handle the entire brick. Though I'm sure you already have already thought this part through, so ignore me if I am totally off base.

"Always code as if the person who will maintain your code is a maniac serial killer that knows where you live" -Unknown

"If you've done something right no one will know that you've done anything at all" -Futurama

"Have a nice day, or not, the choice is yours." Tom Steiner

<Michael_V> or create a Borg collective and call it The 83+
<Michael_V> Lower your slide cases and prepare to be silent linked. Memory clears are futile.
TheStorm wrote:
Woo, and with s3tc compression you can make it so that the color map texture doesn't need much vram to use and from what I hear it is rather easy to map the 4*4 patches to the face they need to be on.

Agreed, although I'm considering upping it to 8*8 depending on the requirements of the Torque material system with regards to UV coordinates for normal maps (for material texturing).

TheStorm wrote:
A thought I just had for the studs would be to maybe use a tiled texture of it with an alpha layer on the top of the bricks so that you only need the color map texture and the stud texture to handle the entire brick. Though I'm sure you already have already thought this part through, so ignore me if I am totally off base.

My tricks with the primitives will probably need to be a little bit more sophisticated, but that sounds okay. I'll probably end up making use of some LOD tricks.
<- Please Support on Lego Ideas
Whoa, I had missed a rare post from CyberPrime! Great to hear about these efficiency ideas; efficiency will definitely be excellent for those of our members with weaker computers, now that we're going with the more intensive T3D engine.
Geopipe
  
Register to Join the Conversation
Have your own thoughts to add to this or any other topic? Want to ask a question, offer a suggestion, share your own programs and projects, upload a file to the file archives, get help with calculator and computer programming, or simply chat with like-minded coders and tech and calculator enthusiasts via the site-wide AJAX SAX widget? Registration for a free Cemetech account only takes a minute.

» Go to Registration page
Notes for implementing LDraw Part Spec
» Goto page Previous  1, 2, 3, 4  Next
» View previous topic :: View next topic  
Page 3 of 4
» All times are UTC - 5 Hours
 
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum

 

Advertisement