Hi,
This afternoon I had the idea to look at my 8-year-old TI-BASIC code, written in highschool during long math lessons. I just have a .tig archive from that time.
Short version: is the de-tokenizer available as simple program (e.g. "kermdetoken < PROGRAM.x8p > source.basic")? That would allow me to improve it and it'd be more convenient.
Long version
Inside the .tig are 52 files, and more precisely
15 8xk
9 8xv
9 8xn
4 8xg
3 8xs
3 8xl
2 8xp
2 8xi
1 8xz
1 8xy
1 8xw
1 8xt
1 8xm
I started with gfm (based on TiLP) which was able to list the files & their type in a .tip archive, but nothing else. Then hexdump, z80dasm & related, and finally tilem2 (an emulator) which crashes.
I've uploaded all the files on SourceCoder, but selecting them all together.
An error message could be raised for the first file but I wouldn't get any notification on the status of the others.
Many files were de-tokenized correctly but it failed on several, and some seem a bit amiss.
Therefore I was wondering if the source of SourceCode was available somewhere.
Thank you for your work!
Welcome to Cemetech! I hope that SourceCoder will be able to help extract the various files there. You may or may not have realized that a .tig is really just a .zip file, so you don't need any special program to get the files out. Anyway, it sounds like you are past that part, so let's focus on the files that did and did not work. The tool is unfortunately closed-source due to bad experiences in the past, but I'm more than happy to see what I can do to help with the files that didn't work. Of those, I expect that SourceCoder will be able to read:
Code: 15 8xk (App) -> No (Apps are ASM)
9 8xv (AppVar) -> Some (AppVars may contain human-readable data)
9 8xn (Real) -> Yes
4 8xg (Group) -> Depends what's inside
3 8xs (String) -> Yes
3 8xl (List) -> Yes
2 8xp (Program) -> Some (TI-BASIC only. ASM programs cannot be detokenized)
2 8xi (Picture) -> Yes
1 8xz (Zoom Settings) -> No
1 8xy (Y-Equations) -> No
1 8xw (Window) -> No
1 8xt (Table Settings) -> No
1 8xm (Matrix) -> Yes
Hi,
Thank you for the quick reply. I'm actually interested in the 8xp files, and the 8xg files which contain several 8xp-style programs.
Amongst the strange things, here are some de-tokenized lines:
Code: If 4=abs(B+[A](C+1,D)+[A](C+2,D)+[A](C+3,D:End:Goto theta
Disregarding the unclosed parenthesis (IIRC execution was quicker that way...), is it normal that there's no linefeed? Another problematic line is
Code: DelVar ADelVar BDelVar CDelVar DDelVar EDelVar FDelVar GDelVar HDelVar [A]DelVar L1prgmA
Another line that seems strange:
Code: randInt(1,4->|LMAS(A
What's that "|LMAS"? It appears in multiple places where I'd expect a list. Is that correctly de-tokenized?
Finally when I try to save a project or open it in jsTIfied I get a "Data too large" message and cannot check whether it works...
bru wrote:
Amongst the strange things, here are some de-tokenized lines:
Code: If 4=abs(B+[A](C+1,D)+[A](C+2,D)+[A](C+3,D:End:Goto theta
Disregarding the unclosed parenthesis (IIRC execution was quicker that way...), is it normal that there's no linefeed? This is normal. The colon is the equivalent of a linefeed. bru wrote:
Another problematic line is
Code: DelVar ADelVar BDelVar CDelVar DDelVar EDelVar FDelVar GDelVar HDelVar [A]DelVar L1prgmA
DelVar is a special command; either due to an error by TI or an intentional feature, you don't need a newline or colon between DelVar and the following command.
Quote:
Another line that seems strange:
Code: randInt(1,4->|LMAS(A
What's that "|LMAS"? It appears in multiple places where I'd expect a list. Is that correctly de-tokenized? |L is jsTIfied's representation of the small L used to preface custom lists. It's a relatively standard representation among non-TI TI-BASIC editors.
Quote:
Finally when I try to save a project or open it in jsTIfied I get a "Data too large" message and cannot check whether it works...
Does that appear as a popup that complains about DOM storage space, or a message in a red box at the top of SourceCoder 3?
Thank you for your help.
Quote:
Does that appear as a popup that complains about DOM storage space, or a message in a red box at the top of SourceCoder 3?
Message in a red box at the top!
*necrobump* In my ongoing quest to be able to declare SourceCoder 3 no longer a beta piece of software, I have implemented half of a long-requested feature: indentation. Now, SourceCoder will automatically add indentation to TI-BASIC code while you type, and will strip out indentation before exporting programs as .8xp. Coming soon: Axe indentation, and indentation of imported programs.
For those having trouble, It doesn't indent the line following the If statement. However, if you do If:Then, the next line is indented. Whereas if Then is on a new line, next line (Then in this case) is not indented.
Indeed, I felt that that was the least confusing behavior. I don't generally consider If followed by a single conditional statement as a tabbable control block, whereas blocks that end in End seem like they should be. Anyway, updates:
- Imported .8xp and .8xv files are now indented as well
- User configuration items are needed, and will be added soon: (1) whether to indent at all; (2) whether to use tabs or spaces for indentation
I am working on a program that uses a string containing several tokens - like "For(While 123-Repeat "->Str0. When I upload the program to SC3, the code after the string is indented.
edit:
On one of my computers it seems that indentation is always on, and cannot be turned off, I also cannot delete indentation. This also happens when I'm not logged in or using a different browser.
Nice feature. I wonder if existing projects will be automatically indented in the near future?
joblo wrote:
I am working on a program that uses a string containing several tokens - like "For(While 123-Repeat "->Str0. When I upload the program to SC3, the code after the string is indented.
Yep, unfortunately the non-stateful detokenizer doesn't know when it's inside a string for indentation. That's part of why I need to add the option to disable indentation.
Quote:
On one of my computers it seems that indentation is always on, and cannot be turned off, I also cannot delete indentation. This also happens when I'm not logged in or using a different browser.
That's correct; it will keep re-adding the indentation. I will add the ability to disable indenting shortly.
Maybe we could have two space indentation as an option too? I've always felt that 4 spaces (a tab) of indentation is too much.
CKH4 wrote:
Maybe we could have two space indentation as an option too? I've always felt that 4 spaces (a tab) of indentation is too much.
Yes, the options I'm currently planning to add are:
1) Whether to indent imported files and typed code.
2) Whether to use spaces or tabs
3) Tab width, in spaces.
Ah that is great to hear. Good luck on finishing this.
*bump* Indentation now:
1) Is toggleable, both for uploaded and typed programs
2) Can be set to use tabs or spaces
3) Allows a user-specified tab width from 1 to 16 characters
You can set all of these settings from the Help/Settings area of the SC3 main menu.
Imported files are still indented following a String containing certain key tokens. I would disable indentation, but then I would have to indent all of the legitimate things by hand. Maybe this could be fixed by ignoring things on a line that begins with " when parsing for indentation?
Edit: I can also not manually add indentation after a certain point in my program.
joblo wrote:
Imported files are still indented following a String containing certain key tokens. I would disable indentation, but then I would have to indent all of the legitimate things by hand. Maybe this could be fixed by ignoring things on a line that begins with " when parsing for indentation?
The problem is that imported programs are indented on the server side, which uses a different (non-stateful) indentation algorithm than the client-side algorithm applied to typed code. I can probably fix this by leveraging the client to indent imported code. I'll try to get to this in the next N days.
joblo wrote:
Edit: I can also not manually add indentation after a certain point in my program.
Can you be more specific? Is this with automatic indentation enabled or disabled? Does it happen after a specific line number in every program? What do you type, what do you expect to happen, and what happens instead?
Here's a bug report for myself: When you open a project, the first character of the program disappears.
KermMartian wrote:
joblo wrote:
Edit: I can also not manually add indentation after a certain point in my program.
Can you be more specific? Is this with automatic indentation enabled or disabled? Does it happen after a specific line number in every program? What do you type, what do you expect to happen, and what happens instead?
I can now not add any indentation anywhere in the program using Tab or Space. When I press Tab in front of a line that already has indentation, the indentation is removed. If there is no preexisting indentation, nothing happens. When I type a new line inside of a control structure, it is not indented automatically even with indentation turned on. The same problems exist with indentation disabled.
When I do have a standalone "Pause ", it don't get converted as one token, but the tokens "P", "a" etc.
PT_ wrote:
When I do have a standalone "Pause ", it don't get converted as one token, but the tokens "P", "a" etc.
Use Pause + a Space. Then it will convert to a Token.