Enno Bartels
Super tuxi


Tool for diff, html, hex edit

multidiff - version 0.0.9 (22. aug. 2002)

Multidiff

- About

- Features

- Options

- Screenshots

- Download

- Mail

- Ini file example

- Info file example

- Todo


Back

* About:

m u l t i d i f f:
This pogramm with the name multidiff was developed for analysing binary data files.

If you have to reengineer a unkown fileformat you have to look for differences in the file data and then you can understand what is happening there. Of course is it a difficult way to check the differences. An other problem was that the normal diff programm can not give you an output. It tells only that the binary data files differ. There are ways to do that but you have to do this and that before there is what you need.
And you often need a html output to share the information with other developer.


q m u l t i d i f f:
Because there are so many options for multidiff I have written a qt frontend - called qmultidiff. It is only used to select the used options. You can save the options in a xml file.
qmultidiff is part of multidiff.
If you do not have qt on your machine qmultidiff will not be compiled but multidiff will.





* Features:

Output modes
     - output can be a html or a text file.
     - output can be done into a file or to standard output.

Visu row modes
     - you can diff the files in different visualisations rows:
         - ascii
         - hexadecimal
         - decimal
         - octal
         - bit (each bit is compared)
     - you can see the position on a visualisations file-position-row.
     - you can change the order of the different visualisitions rows.

Information and statistics
     - you can add a ascii file with hexadecimal value and the explaniation of that as strings. So you get the information visualisation. The lenght of the strings is automatic adapted to the length of the values of the hexadecimal values. row.
     - you can get an statistics output table for the hex-string. informations

Diff
     - you can diff with more than 2 files.
     - the different values between to files are colored.
     - you can start a automatic diff and the line will be moved in right place.
     - you can give also manually diff positions and the values will be moved to that place.
     - but some problems are still there.
           List of tested different diffing algorithms
           1. Analyses of the diff work with 4 files
           2. Analyses of the diff work with 4 files
           3. Analyses of the diff work with 4 files with other algorithm
Else
     - you can hide identical start or stop ranges of the files.
     - you can give manualy the start und stop position to look on.
     - you can tell how much spaces should be between 2 values.
     - you can change the colors, the fonts, the look very much.
     - you can add a .multidiff initialisation file in your home directory where you can put all the options you are using by your own default. The ini file is using the same options strings like that you did use on the command line






* Screenshots:

     1. A simple example (2 files)

     2. A simple block example (2 files)

     3. Hide equal start values (2 files)

     4. The Rows: ascii bit hexadecimal decimal (2 files)

     5. The info mode (2 files)

     6. The block mode (with and without linking) (2 files)

     7. Different ascii and row posibilties (2 files)

     8. The row ordering plus start and stop position (2 files)

     9. The moving diff mode (2 files)

     10. The moving diff mode (more then 2 files)
         Is not really ready jet!!

     11. The info statistik (simple and detailed)
         Detailed is not ready jet!!

     12. Filelist and block list with and without links
         Image will comeing soon!

     13. Vertical mode
         Is not ready jet!!

     14. Output into multiple html pages
         Is not ready jet!!






* Initialisation file example:

-wsasc p
-wsinf '#'
-col "FF0000 FF00FF 00FFFF FFFF00 00FF00 "






* Infofile file example:

One line with the hexadecimal values. A -1 is used to indicate the end of the hexadecimal values line.
You can also use a -9 instead of a hexadecimal value for one unknown value.

 01  ff  02  04  03  02  00  02  -1
Header_part_1
 
 21  05  00  04  01  ff  03  ff  00  -1
Header_part_4a
 
 10  00  00  02  19 -1
Header_part_6
 
 -9  -9  01  18  -9  08  1a  -9 -1
cm_or_zoll

 c8  00  01  18  04  08  1A  01 -1
Header_part_6b_(cm)






* Options:

usage: ./multidiff [Options] <file1> <file2> ..

 Options:
 --------
 * = default

   -help    =   Prints this full list with all options.

   -v       =   Prints the actual version of multidiff and exit.

   -c x     =   Number of data values in one line
                (default: the whole file in one line)

   -ufna    = * Use file names   for the diff table
   -ufnu    =   Use file numbers for the diff table
                The file numbers option is only useful together
                with the file list option (-fl)

   -o x     =   Give the output filename.
                (default: for html-mode: <result.html>
                          for txt-mode : <result.txt> )

  Modes:
  - - - -
   -vm      =   Vertical   display mode (not useful yet!!)
   -hm      = * Horizontal display mode

   -ono     =   Write no output
   -ofil    = * Write (html/txt) output info file
   -ostd    =   Write (html/txt) output info stdout

   -ohtm    = * Write as html output
   -otxt    =   Write as text output without frames
   -otxt2   =   Write as text output with frames

  Rows:
  - - -
   -ahl     = * ASCII html output for the first file
   -nahl    =   No ASCII html output for the first file

   -abl     =   ASCII html output at the other files for diffing
   -nabl    = * No ASC html output at the other files for diffing

   -d       =   Decimal html output
   -nd      = * No decimal html output

   -7       =   Octal html output
   -n7      = * No octal html output

   -h       = * Hexadecimal html output
   -nh      =   No hexadecimal html output

   -b       =   Bits html output
   -nb      = * No bits html output

   -p       = * File position html output
   -np      =   No file position html output

   -fprd    = * File position row as decimal values
   -fprh    =   File position row as hexadecimal values

   -m       =   Memory position html row.
   -nm      = * No memory position html row.
                The option (-m) is very useful in the diff mode
                (-mh  and  -md)

   -inf x   =   Set a file as input for regex values
                This is used for the info rows in the first file.
                So you can get an explanation what the hex-code
                means.
                x = Name of the file with regex infos
   -ninf    = * Do not set a file as input for regex values

   -infa x  =   Set a file as input for regex values
                This is used for the info rows at the all files
                (first file and the other files).
                So you can get an explanition what the hex-code
                means.
                x = Name of the file with regex infos
                (default: Not activated)

   -mprd    = * Memory position row as decimal values
   -mprh    =   Memory position row as hexadecimal values
   -prv     =   Vertical pos row mode: each char of
                a pos value will be displayed one
                below the other.
                (default: Not activated)

   -cr      = * Cut row html output
   -ncr     =   No Cut row html output

   -pref    =   Position row in each file
                (default: Not activated)

   -ordh x  =   Order of the html rows (only the header)
                (default:  -ordh piadohb)

   -ordb x  =   Order of the html rows (only the body)
                (default:  -ordb piadohb)

   -ooci    =   Length of the information text  (row/column) is
                only 1 character. (hex) -> (h)
                (default: all characters)

   Order Information:
                - p  for pos (this works only with the md-mode)
                - i  for info
                - a  for asc
                - d  for dec
                - h  for hex
                - b  for bit
                - o  for oct
                Example :  -ord adhibp
  Colors:
  - - - -
   -crcol x =   Background color of the cut row
                (default: <000000> )

   -mmcol x =   Background color of the global memory row
                (default: -mmcol DDDDDD )

   -fpcol x =   Background color of the global file position row
                (default: -fpcol DDDDDD )

   -dfcol x =   Color of the different expression
                Only if the first value is existing
                (default: -dfcol FF0000 )

   -dfcol2 x=   Color of the different expression
                Only if the first value is not existing
                Only in -md mode
                (default: -dfcol2 FF8800 )

   -bccol x =   Background color of the block counter column/row
                (default: -bccol DDDDDD )

   -bgcol x =   Background color of the webpage background
                (default: -bgcol FFFFFF )

   -brcol x =   Background color of the reg text elements
                (default: -brcol F0F020 )

   -col "x" = Background colors of the files (1 or more)
              example:  -col "FFFFFF DDDDDD EEEEEE 543452"
              (default: -col "FFDDDD DDFFDD DDDDFF" )

  Start - stop:
  - - - - - - -
   -nequs   =   Hide an identical start range
   -equs    = * Do not hide an identical start range
                The option -nequs is not allowed with -s

   -neque   =   Hide an identical end range
   -eque    = * Do not hide an identical end range
                The option -neque is not allowed with -e

   -s xx    =   Set the start byte number to display
                This option is not allowed with -nequs !!!
                (default: 0)

   -e xx    =   Set the end byte number to display
                This option is not allowed with -neque !!!
                (default: last byte of the biggest file)

  Text output:
  - - - - - - -
   -txtwc   =   Use colored diff text in text mode
   -ntxtwc  = * Do not use colored diff text in text mode

   -hf      =   Generate from each input file one output file
                with the binary input data as ascii hex data
                and with only one value per line
                (default: Not activated)

  Statistics table:
  - - - - - - - - -
   -st      =   Show statistics on html page
                (default: Not activated)

   -stpt    = * Statistics position on top of html page
   -stpb    =   Statistics position on bottom of html page

   -stwl    =   Show statistics with links
                (default: Not activated)

   -stfna   = * Statistics: Use file names
   -stfnu   =   Statistics: Use file number
                Use file number is only useful together with the
                file list option -fl

  File table:
  - - - - - -
   -fl      =   Print a list of all files first
   -nfl     = * Do not print a list of all files first

  Block counting:
  - - - - - - - -
   -bc      =   Block counting
   -nbc     = * No block counting

   -bcl     =   Block counting linking info on page
   -nbcl    = * No block counting linking info on page

  Diff:
  - - -
   -md      =   Move diff visualize option
                -pref is automatically set
   -nmd     = * No move diff visualize option
                (default: -nmd)

   -mh x y z=   Move diff visualize option by hand
                x=filenumber y=filepostion z=movewidth
                This option can be added multiple times
                (default: Not activated)

   -pmlmh   =   Print the move elements like move hand elements
                (default: Not activated)

   -pmeof x =   Print the move elements in the file x
                (default: Not activated)

  Width:
  - - - -
   -trsm    =   Try smallest value width (0)
                If a value is wider it will always use the
                widest width!
                (default: 3)

   -ec x    =   Number of whitespaces between two values
                (default: 1)

  Look:
  - - -
   -wsasc x =   Whitespace for a not showable character at the
                ascii row.
                (default: < >)

   -wsinf x =   Whitespace for a unknown value at the info row.
                (default: < >)

   -fsfn x  =   Font size of the filenames. (-5 ... +5)
                (default: 0)

   -fstd x  =   Font size of the type description. (-5 ... +5)
                (default: 0)

   -fsda x  =   Font size of the data range. (-5 ... +5)
                (default: -1)

   -tbcol x =   Table border color
                (default: DDDDDD)

   -tb x    =   Table border width
                (default: 2)

   -tcs x   =   Table cell spacing
                (default: 0)

   -tp x    =   Table padding
                (default: 5)

   -pt x    =   Page title: Title of the window.
                (default: empty)

   -pa x    =   Page author .
                (default: empty)

   -pe x    =   Page author - email address.
                (default: empty)

   -ph x    =   Page title: Title of the page.
                (default: empty)

  Debug:
  - - - -
   -deb x   =   Print some debug information
                It's a bit code:
              * -  0 = don't print debug infos
                -  1 = print only  reg-tag         debug infos
                -  2 = print only  detailed diff   debug infos
                -  4 = print only  statistics      debug infos
                -  8 = print only  general diff    debug infos
                - 16 = print only  memory alloc    debug infos
                - 32 = print detailed memory alloc debug infos
                example : -deb 3


   -sizes   =   Print the sizes of all structures and exit.











* Download:

22. aug 2002: multidiff-0.1.0.tar.gz

17. feb 2002: hex_diff-0.0.98.tar.gz
10. feb 2002: hex_diff-0.0.97.tar.gz
22. jan 2002: hex_diff-0.0.95.tar.gz
11. jan 2002: hex_diff-0.0.9.tar.gz

configure
make
make install







* Mail:

Feel free to send comments/suggestions/patches !
ennobartels@t-online.de





* Todo:

         

new:
-----
 - free at moveItems segfault ??
 - change the moveItem structure to moveItem
 - bug: -s-e option real block counter !!!
 - text mode: color for diff: -tdcol 31  -tdcol2 33
 - rebuild the move diff function. with diff3 or anything else ????
 - bcl should be possible without bc and should be moved to -wl (with links)
   this is for -fl and -bc if on is -wl then okay
 - add -ms and -me for memory start and stop position

Output:
-------
 - output into different files
   - only possible in html mode
   - -ohf0 output html file mode in one file (default)
     -ohf1 output html file mode in 3 files: index, statistic, diff
     -ohf2 output html file mode in x files: index, statistic, diff, each block
     ->fpout[] as array dito name of the output files names
   - onb x  number of blocks per html page


PROG_TEST listing
-----------------
   - add  check_moveHand, sort_moveHand, swap_moveHand, move_handProceed,
   - add  moveItem_auto
   - add  print_size
   - add  row_head, row_foot,
   - add  re_mem, ma_mem
   - add  get_diffItems
   - add  moveHand -> moveItems

Rows:
-----
 - short int row:
   - [n]rsi0 -> floating point row output (short int = 2Bytes) with offset 0
   - [n]rsi1 -> floating point row output (short int = 2Bytes) with offset 1
 - float row:
   - [n]rf0 -> floating point row output (float = 4Bytes) with offset 0
   - [n]rf1 -> floating point row output (float = 4Bytes) with offset 1
   - [n]rf2 -> floating point row output (float = 4Bytes) with offset 2
   - [n]rf3 -> floating point row output (float = 4Bytes) with offset 3
 - double row:
   - [n]rd0 -> floating point row output (float = 8Bytes) with offset 0
   - [n]rd1 -> floating point row output (float = 8Bytes) with offset 1
   - [n]rd2 -> floating point row output (float = 8Bytes) with offset 2
   - [n]rd3 -> floating point row output (float = 8Bytes) with offset 3
   - [n]rd4 -> floating point row output (float = 8Bytes) with offset 4
   - [n]rd5 -> floating point row output (float = 8Bytes) with offset 5
   - [n]rd6 -> floating point row output (float = 8Bytes) with offset 6
   - [n]rd7 -> floating point row output (float = 8Bytes) with offset 7
 - Visualisation
   with %08.4f
   of with:  o's:0|1  number of digits:x  number of digit behind the point:y
 - add a formel for example for zooming

ini-file:
---------
 - add error message for seach option
     0 -> error from inifile
     1 -> error from commandline
 - ignore ini file when the -ii comes on the commandline!
 - add the slash for ini file options: -wsinf \"  -wsinf ' ' -wsinf " "
 - check again: del_comments, read_iniFile, get_noElements: reduce code

Screenshots and desciption images:
----------------------------------
 - describe with images : -bc -bcl -fl whats happening then ?

Bugs:
-----
 - Is there a netscape bug at 1000 characters in one row with the table lines
 - xterm bug ??Problem with -otxt and -txtwc if linelenght to long -> linebreak
 - maybe: Start and stop at the right place use an array with the number of
   files start[] stop[]


Statistic:
----------
 - with detail statistic not only counting them: make a list with each pos in
   the file
 - maybe with a link
   a name="reg01_file01_01"  (no of the reg, no of the file, no of the existing
 - move the reg info loading out of stat to something before
   stat_html sould only show what WAS fould


Color:
------
 - option for the info row text background color
 - pos-color with a little grew at md mode: -g,  -ng,  -gf faktor
   get maxcolor from r, g or b and add grew then .. divide throught maxcolor
 - Is there any nice filling char for the empty spaces at the diff mode

Diff md:
----------
 - Diff md: Change info row from %s to %c for the -c mode and tag splitting
 - Diff md: Use another algorithmen, maybe switching by the user
 - Diff md: Color of b, d mode if the value is not existing
 - diff 2 files then diff a new file with the 2 existing and look at items
   try that in therory

Nice to have:
----------
 - vertical display mode -vm
     ini.mode = 1 = vertical 2= horizontal
     Example 1:
      File 1        |   File 2
      --------------+----------
      8a  10101111  |   7b 11001010

     Example 2: (-v2)
      File 1        |   File 2
      --------------+----------
             8   1  |  7  1
             a   0  |  b  1
                 1  |     0
                 0  |     0
                 1  |     1
                 1  |     0
                 1  |     1
                 1  |     0
 
    for (a=0; a<8; a++) pos_col (a, ...)
                        hex_col (a, ...)
                        {
                          if (a>1) return
                        }
                        bin_col (a, ...)
                        {
                          if (a>7) return
                        }
 
 - Filename "/bla/bla/mwtext.drw"  to "/bla/....mwtext.drw"
   maybe: files with identical pathes could be shorted and writting only
   once -pfsh
 - Filename list bevor start of prog
 - Filenames each time of row use -fner  -nfner
 
 - add a ini file in commandline ? -ini xxx
 - commandline option : do Not load the Ini File in your Home .multi_diff -nifh
 
 
Rest:
-----
 - If a regtag is on the end of the line: right splitting
 - Different visualisations for the diffing (bold, italiq)
 - Option -searchd "" give position and counted element as static
   with links to the places inside the file
 - show only diff places. Hide everything else ?? -shod
 - Flexibel eof char -ceof
 - Position row:
   add a startvalue. With this the values is not able to get wider and wider
 - Instead of -c xx use -cb for the number of blocks the programm should show
 - check html code with validate web
 
 
Not usefull:
------------
 - check dobble commandline expressions (no thats not usefull)
 - add bgimage ??