Captain Calc wrote:
When this code is run, it does not produce any output for the TI_PRGM_TYPE. So far the source of this bug has eluded me. Any thoughts?

I only skimmed the code, but it appears you're not resetting your VAT seek every time you change the type of var you're looking for.
You probably want the following line of code inside your outer-most for() loop:

Code:
void *detect_str = 0;
Yes, that was it! Thank you so much!
Great progress on this! I noticed you talked earlier about a “headless start” system, and I’d like to have VYSION support that. Will it just need to store the file name in Ans, and how will you differentiate between appvars and programs of the same name?

Looking forward to seeing more updates on this, so I have a few more suggestions as well (some a restatement of things I’ve already said):
-being able to press [enter] to select things
-differentiating between protected TI-Basic and ASM programs (check the start of the file for 0xef7b, and it’ll be ASM)
-customizable color schemes (?). If you’d be interested in making this program enhanced with VYSION later on as well (that functionality has yet to be announced), there will be an include file you can use to get these colors, to have the same appearance as the shell.
-load all programs and appvars before creating the menu in a dynamic array of structs that stores the name, type, and size of the program. It appears to me, based on this code that every loop, the files are redetected, which is needless overhead and results in the list scrolling more slowly towards its end. If you just load all this beforehand by detecting the files at program start, you’ll get consistent and faster speed regardless of the number of programs or the offset in the list.

That’s all I have right now, keep up the work on this great project!
Thanks for the feedback! Smile

Where did you notice the lack of [enter] support? I believe I carried all of dual button support over from the 1.2.2 version, but there could be a spot I missed. I don't think color customization is a high-priority issue right now because I want to keep the program from growing too much further, but I am considering adding additional configuration data to the Headless Start that will allow other programs to override the default color scheme. I like your idea of differentiating between the different kinds of protected programs and the pre-loading of the file data into dynamic structure arrays. The main menu cursor speed was always an Achilles' heel for HexaEdit.

I forgot to mention that this version also supports a theoretically infinite number of undos. The actual number is roughly 5000, but this should be ample unless you are writing an entire assembly program using hexcodes. Razz
Progress Update

I have finished basing the main menu backend on the dynamic file data arrays and am working on rewriting the functions that manage the Recent Files list. The Headless Start now supports a "color theme" override that allows the calling program to decide how the editor will look. The configuration data for the Headless Start is split up into four structures: a general header, a color theme override, a memory (RAM/ROM) editor configuration, and a file editor configuration. I will include a guide to the Headless Start API in the program download.

For the first release at least, HexaEdit will use an appvar to store the Headless Start data. I did some experimenting with the Ans variable but did not figure out how to store non-integer values (pointers, etc.) to it. Looking at the documentation for Ans, it looks like it can only store integer values, unless KryptonicDragon found some hacky way to make it hold other data. Wink

Unless something unusual occurs, I hope to release version 2.0.0 in the next couple of days! Smile
Version 2.0.0 Release

HexaEdit 2.0 is now available for download!

This version features a reworked GUI for both the main menu and editor, an upgraded file search that can search both appvars and programs, and a Headless Start API that allows the program to be run from other C or ASM programs.

Many thanks to all those who encouraged HexaEdit's development and helped improve it! Smile
Looks pretty good indeed, and the feature set is definitely nice - keep up the good work Smile
It reminds me of Command Post Plus and TICT's Tools Suite handle explorer (tthdex), both for the TI-68k series.
Thanks a lot! Smile

I am hoping to release a small update to HexaEdit tomorrow that will fix a bug in the Headless Start routine, make some minor tweaks to the input system, and fix some memory management issues in the main menu. I have at KryptonicDragon's request added a switch for the offset/address column on the left-hand side of the editor that allows the user to switch between offsets (calculated from the start of the file, RAM, etc.) or RAM/ROM addresses. The [mode] button toggles this switch.

HexaEdit 2.0.1 also offers a feature suggested by Lionel Debroux: a phrase finder for the editor. This function allows the user to search for a 16-character phrase in ASCII characters or an 8-character phrase in hexadecimal. This function is very fast for files, reasonable in RAM, and very slow in ROM (~90 seconds to find all occurances 2-character hexadecimal string in over 4 million bytes). Although it does remove the satisfaction to know that there are exactly 3 occurences of "\xab\xcd" in the entirety of ROM, I will probably cap the search range to around 300K bytes in order to provide a decent response time unless enough votes come in to make it an option. Wink
Version 2.1.0 Release

Download: https://www.cemetech.net/downloads/files/2010/x2384

This version introduces a shortcut to a file's VAT location and makes the phrase search algorithm 6x faster than the previous version! Now, when the user selects a file in the main menu and presses [mode], HexaEdit will open a RAM editor at the file's VAT location, saving them the trouble of remembering the VAT address, opening the RAM editor, and typing it into the Goto function.

I rewrote the phrase search code in assembly and now it only takes a little under 15 seconds to search the entirety of ROM instead of the 90 seconds the C algorithm took. Searching all RAM is even faster: only 1 second! This performance boost made the custom search range superfluous, so I removed it but left the option for other programs set custom search ranges by modifying the settings appvar.

Hope you enjoy it! As always, any feedback would be greatly appreciated! Smile
It looks great except that the file scrolling is really slow! Maybe if the program didn't update the time every time a file was scrolled it would be faster. Also, since the primary users of this program are developers, and developers' calculators' ram usually resets pretty often, you might think about just cutting the time viewer entirely. However, I use this program all the time and it is really great otherwise!
Thanks for the comment! Smile

Removing the clock sounds like a good idea. All the shells for the CE have clocks, so HexaEdit's clock is sort of redundant. You made a good point about the constant RAM resets, too. Rolling Eyes Setting the clock back up after each reset is a hassle.
Captain Calc wrote:
Thanks for the comment! Smile

Removing the clock sounds like a good idea. All the shells for the CE have clocks, so HexaEdit's clock is sort of redundant. You made a good point about the constant RAM resets, too. Rolling Eyes Setting the clock back up after each reset is a hassle.


Since your removing time displaying, what would you be placing in the empty space "where the time used to be"?
Just nothing would be fine, I think. One thing I would say though is that as I said earlier, the call to get the OS battery status is very slow (the clock too, most likely). Just having it run every 30s or so would fix the problem, I think.
epsilon5 wrote:
Just nothing would be fine, I think. One thing I would say though is that as I said earlier, the call to get the OS battery status is very slow (the clock too, most likely). Just having it run every 30s or so would fix the problem, I think.


I would not display the battery at all, and center the program's name. Also, this would dramatically increase the program's speed.
Alvajoy123: I had not thought about replacing the clock with anything, but I am toying with the idea of using the space for extra versioning information, like the Git branch for the build.

Alvajoy123 and epsilon5: I made a test build without the battery indicator and it greatly improved the cursor speed. With a few tweaks to slow down the table tab switching, the main menu will be much more responsive than the last release.

I have also implemented a "superuser" lock that prevents the user from writing to certain RAM address that would crash the calculator. To toggle the superuser mode, you press the [^] key while in the editor.

The "Ports" area of the calculator (0xE00000 - 0xFFFFFF) is now supported, per jacobly's request. I am busy rewriting sections of the editor code that have potential variable overflows because of the unique upper bound of the "Ports" section. Note: It is possible to crash/freeze the calculator by writing to certain bytes the "Ports" area.

Thanks for the comments! Smile
Version 3 in Development
As some of you may already know, I am working on a third version of HexaEdit CE that will improve the program's stability, add some new features, and have better source code documentation.

A few of the issues that v3 will fix are:

  • The overall sluggish response time in both the main menu and and in the editor
  • Empty file handling (a bane of the previous versions)
  • Lack of a testing framework for edge cases


The features that I am hoping add in v3 will be:

  • The superuser lock (mentioned in the last post but extended to the "Ports" area)
  • Password for the superuser
  • Cut-Copy-Paste functions for the editor
  • The option to write either nibbles or entire ASCII characters
  • A rebooted Headless Start system that will allow TI-BASIC/ASM/C programs to open editors or to edit memory without opening an editor
  • The "Ports" editor
  • An editor for TI-vars, such a reals, strings, etc.
  • Possibly an option to switch between the v1 and the v2 main menu styles


This version will be a complete rewrite in C++ and ASM.


Software Design Questions

HexaEdit is now split into three parts: the core (handles the actual memory operations, reading, writing, etc.), the GUI (main menu, editor, etc.), and the Headless Start. In the core is three abstract base classes: ABC_ReadOnlyMemory, ABC_ReadWriteMemory, and ABC_VariableMemory. Each ***Memory class describes a section of memory and its attributes. ABC_ReadOnlyMemory would describe ROM and ABC_ReadWriteMemory would describe RAM, for example.

ABC_ReadOnlyMemory implements all reading-related functions. ABC_ReadWriteMemory inherits from ABC_ReadOnlyMemory and adds writing functionality to it. ABC_VariableMemory inherits from ABC_ReadWriteMemory, is only used for files, and adds byte insertion/deletion functionality.

Each of these abstract base classes (ABC's) do not have any sanity-checking and assume that all of the arguments passed to them are valid. These classes can be considered the core's backend. Needless to say, these classes cannot be accessed directly, but are instead each encapsulated by a derived class: ReadOnlyMemory, ReadWriteMemory, and VariableMemory. These classes mirror all of the member functions in the base class and sanitize the arguments before passing them to the ABC for processing.

This setup makes sense to me, but I wanted feedback from the other, more experienced programmers on Cemetech who have designed similar systems. Will this design yield a robust core? Will it be maintainable and, if need be, extensible?


Another question that I had related to debugging. Consider this function from ABC_ReadOnlyMemory which finds all occurrences of a given byte sequence in a particular memory area:

Code:

uint8_t ABC_ReadOnlyMemory::FindAll(
  uint8_t **matches,
  const uint8_t max_matches,
  const uint8_t seq[],
  const uint8_t len,
  const uint24_t offset,
  const uint24_t range
)
{
  uint8_t num_matches = 0;
  uint24_t range_from_start;


#if DEBUG
  CCDBG_CHKPT_START("ABC_ReadOnlyMemory::FindAll");
  CCDBG_DUMP_PTR(matches);
#endif

  // Case 1: (<offset> + <range>) <= memory size
  // The <range> can be wildly irrational, but the <offset> will be less than
  // <m_Size>, thus this comparison must be written so:
  if (range <= (m_Size - offset))
  {

#if DEBUG
  CCDBG_PRINT_MSG("Case 1");
  CCDBG_DUMP_PTR(m_Addr + offset);
  CCDBG_DUMP_PTR(m_Addr + range);
#endif

    num_matches = asm_BFind_All(
      m_Addr + offset,
      m_Addr + range,
      seq,
      len,
      matches,
      max_matches
    );
  }
  else
  {
    // Case 2: (<offset> + <range>) > m_Size

#if DEBUG
  CCDBG_PRINT_MSG("Case 2");
  CCDBG_DUMP_PTR(m_Addr + offset);
  CCDBG_DUMP_PTR(m_Addr + m_Size);
#endif

    num_matches = asm_BFind_All(
      m_Addr + offset,
      m_Addr + m_Size,
      seq,
      len,
      matches,
      max_matches
    );

    range_from_start = range - (m_Size - offset);

    // Case 2a: <range_from_start> < <offset>
    // Loop around to the start of the memory and end before the <offset>.
    if (range_from_start < offset)
    {

#if DEBUG
  CCDBG_PRINT_MSG("Case 2a");
  CCDBG_DUMP_PTR(m_Addr);
  CCDBG_DUMP_PTR(m_Addr + range_from_start);
#endif

      num_matches += asm_BFind_All(
        m_Addr,
        m_Addr + range_from_start,
        seq,
        len,
        matches,
        max_matches
      );
    }
    else
    {
      // Case 2b: <range_from_start> > <offset>
      // Loop around to the start of memory and end at the <offset>.

#if DEBUG
  CCDBG_PRINT_MSG("Case 2b");
  CCDBG_DUMP_PTR(m_Addr);
  CCDBG_DUMP_PTR(m_Addr + offset - 1);
#endif

      num_matches += asm_BFind_All(
        m_Addr,
        m_Addr + offset - 1,
        seq,
        len,
        matches,
        max_matches
      );
    }
  }

#if DEBUG
  CCDBG_CHKPT_END;
#endif

  return num_matches;
}


When built with "make debug" and run inside of a function that tests it for edge cases, it produces the following CEmu console output:

Code:

|
    | main()
    |-----------------------------------------------------------------
    |
    |    | ABC_ReadOnlyMemory::FindAll()
    |    |-----------------------------------------------------------------
    |    |  matches                                           = 0xd1a530
    |    |  Case 1
    |    |  m_Addr + offset                                   = 0x1b0015
    |    |  m_Addr + range                                    = 0x1b7b1c
    |    |-----------------------------------------------------------------
    |
    |  num_matches                                       = 3
    |  data[data_num]->m_NumMatches                      = 3
    |
    |    | ABC_ReadOnlyMemory::FindAll()
    |    |-----------------------------------------------------------------
    |    |  matches                                           = 0xd1a530
    |    |  Case 1
    |    |  m_Addr + offset                                   = 0x1b0015
    |    |  m_Addr + range                                    = 0x1b7b1c
    |    |-----------------------------------------------------------------
    |
    |  num_matches                                       = 172
    |  data[data_num]->m_NumMatches                      = 172
    |
    |    | ABC_ReadOnlyMemory::FindAll()
    |    |-----------------------------------------------------------------
    |    |  matches                                           = 0xd1a530
    |    |  Case 2
    |    |  m_Addr + offset                                   = 0x1b7545
    |    |  m_Addr + m_Size                                   = 0x1b7b1c
    |    |  Case 2b
    |    |  m_Addr                                            = 0x1b0015
    |    |  m_Addr + offset - 1                               = 0x1b7544
    |    |-----------------------------------------------------------------
    |
    |  num_matches                                       = 3
    |  data[data_num]->m_NumMatches                      = 3
    |
    |    | ABC_ReadOnlyMemory::FindAll()
    |    |-----------------------------------------------------------------
    |    |  matches                                           = 0xd1a530
    |    |  Case 2
    |    |  m_Addr + offset                                   = 0x1b7545
    |    |  m_Addr + m_Size                                   = 0x1b7b1c
    |    |  Case 2a
    |    |  m_Addr                                            = 0x1b0015
    |    |  m_Addr + range_from_start                         = 0x1b020e
    |    |-----------------------------------------------------------------
    |
    |  num_matches                                       = 2
    |  data[data_num]->m_NumMatches                      = 2
    |
    |    | ABC_ReadOnlyMemory::FindAll()
    |    |-----------------------------------------------------------------
    |    |  matches                                           = 0xd1a530
    |    |  Case 2
    |    |  m_Addr + offset                                   = 0x1b7545
    |    |  m_Addr + m_Size                                   = 0x1b7b1c
    |    |  Case 2b
    |    |  m_Addr                                            = 0x1b0015
    |    |  m_Addr + offset - 1                               = 0x1b7544
    |    |-----------------------------------------------------------------
    |
    |  num_matches                                       = 3
    |  data[data_num]->m_NumMatches                      = 3
    |  test_succeeded                                    = 1
    |-----------------------------------------------------------------
|


Every programmer has their own preferred method of tracing functions, so this may seem like overkill to some, but I like the verbosity and aesthetic.

Assuming that even a rigorously debugged function can still have bugs that go unnoticed for any number of reasons, I would like to leave the "#if DEBUG ... #endif" blocks in the function so I don't have to rewrite them if another bug crops up in the function. On the other hand, having these blocks of non-algorithmic code makes it harder to follow the function's flow. How could I improve the legibility of the functions while preserving the verbose debugging output?
Version 3 in Beta

In the spirit of Cemetechian "Quality over quantity", introducing HexaEdit 3!



This latest version is a complete rewrite of HexaEdit, discarding a contiguous editor buffer in favor of a dynamic split-buffer design. Its main menu brings back the column interface of version 1 and fixes the slow cursor scrolling that has plagued HexaEdit throughout version 2. Also, for the first time, HexaEdit alphabetically sorts named variables like programs and appvars by itself, instead of relying on a shell to sort the virtual allocation table (VAT).

The program now supports editing of all documented TI-OS variable types (and has an "Other" category if it comes across a variable that hasn't been documented) and that shady RAM segment, 0xE00000 through 0xFFFFFF, the "Ports" area.



On top of the new split-buffer design, the editor offers a direct letter and digit input system that allows users to type alphanumeric characters using the keypad and [alpha] keys, instead of putting in the hex for each character. It also features a global cut, copy, and paste clipboard for transferring up to 255-byte chunks between variables and memory.

The main program is almost complete, awaiting final bug testing. The Headless Start feature from version 2 has yet to be implemented, but it will make its way into version 3, hopefully along with a few new tricks.

That's all for now. Looking forward to sharing this latest version with the community and hope all of you will find it a useful stepping stone to your amazing programs! Very Happy

EDIT: This version is written in good, old-fashioned (and dangerous Razz ) C with a few ez80 assembly routines.
Sounds awesome! That direct letter and digit input sounds particularly useful--there's been more than a few times when I've had to run strings through various converters to get the resulting hex and input it manually into HexaEdit to do what I want. The faster menu scrolling is also a much appreciated change, as I eventually ended up just modifying the source and compiling it myself to fix it, which most users probably wouldn't be willing to do (I think the battery percentage icon/text was causing the issue, I just removed it entirely).

As you're using a split buffer, do you have plans for different input modes (insert/overwrite)? That should be pretty fast now. As I recall, the previous versions only let the user manually insert a specific number of bytes, unless I was missing a setting somewhere. That also would be particularly useful with the direct letter and digit input.

Also looking forward to seeing the Headless Start feature get reimplemented. This is something that I've been planning on implementing into VYSION 2 for a while, but I'm not sure how I should know to use it. Do you just suggest checking to see if the run program is called "HEXAEDIT," or did you have something else planned?

Great work, I look forward to more updates!
Thanks!

Version 3 keeps the battery percentage monitor from version 2 but only updates once every five minutes instead on every editor loop cycle. This delay staves off the performance hit of the slow boot_GetBatteryStatus() call.

I did consider supporting an "insert" mode but scrapped it after considering all of the edge cases (dealing with empty variables, mostly) that the code would have to handle, which would make the program more complex and bigger. However, I might reconsider adding it.

The new Headless Start system is slated to use Ans instead of an appvar like version 2 did, and I want it to look like a standard API call, e.g. VEDITOR("LibLoad", 6) to open a GUI editor for the LibLoad appvar. This is so TI-BASIC programmers can spawn editors, giving a little "scripting" functionality to HexaEdit.

The plan is for shells to load a HexaEdit command into Ans and then use the RunPrgm() function from the toolchain to call HexaEdit. When HexaEdit starts, it will parse Ans to see if it is a valid command, and if so, it will forgo showing the main menu and immediately execute the command. When the user closes the editor, HexaEdit will shut down and return execution to the shell. Is this similar to what you had in mind?
Captain Calc wrote:
The plan is for shells to load a HexaEdit command into Ans and then use the RunPrgm() function from the toolchain to call HexaEdit. When HexaEdit starts, it will parse Ans to see if it is a valid command, and if so, it will forgo showing the main menu and immediately execute the command. When the user closes the editor, HexaEdit will shut down and return execution to the shell. Is this similar to what you had in mind?

Yes, that sounds very reasonable. I think I prefer that to the appvar solution as well, provided that all of the features from the old system are kept (as you deem fit, of course). That ability to allow TI-Basic programmers to use HexaEdit sounds like a great addition as well. What do you have planned for the actual format of Ans? I assume you won't need a whole lot more than variable name and scroll offset.
  
Register to Join the Conversation
Have your own thoughts to add to this or any other topic? Want to ask a question, offer a suggestion, share your own programs and projects, upload a file to the file archives, get help with calculator and computer programming, or simply chat with like-minded coders and tech and calculator enthusiasts via the site-wide AJAX SAX widget? Registration for a free Cemetech account only takes a minute.

» Go to Registration page
Page 2 of 4
» All times are UTC - 5 Hours
 
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum

 

Advertisement