/[gxemul]/trunk/doc/misc.html

This is repository of my old source code which isn't updated any more. Go to git.rot13.org for current projects!

Diff of /trunk/doc/misc.html

Parent Directory | Revision Log | View Patch Patch

-revision 8 by dpavlin,
Mon Oct  8 16:18:19 2007 UTC
+revision 42 by dpavlin,
Mon Oct  8 16:22:32 2007 UTC
 Line 1
- <html><head><title>GXemul documentation: Misc.</title>
+ <html><head><title>Gavare's eXperimental Emulator:&nbsp;&nbsp;&nbsp;Miscellaneous</title>
  <meta name="robots" content="noarchive,nofollow,noindex"></head>
  <body bgcolor="#f8f8f8" text="#000000" link="#4040f0" vlink="#404040" alink="#ff0000">
  <table border=0 width=100% bgcolor="#d0d0d0"><tr>
  <td width=100% align=center valign=center><table border=0 width=100%><tr>
  <td align="left" valign=center bgcolor="#d0efff"><font color="#6060e0" size="6">
- <b>GXemul documentation:</b></font>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
+ <b>Gavare's eXperimental Emulator:</b></font><br>
- <font color="#000000" size="6"><b>Misc.</b>
+ <font color="#000000" size="6"><b>Miscellaneous</b>
  </font></td></tr></table></td></tr></table><p>
  <!--
- $Id: misc.html,v 1.41 2005/06/04 22:47:49 debug Exp $
+ $Id: misc.html,v 1.70 2007/06/15 06:26:20 debug Exp $
- Copyright (C) 2003-2005  Anders Gavare.  All rights reserved.
+ Copyright (C) 2003-2007  Anders Gavare.  All rights reserved.
  Redistribution and use in source and binary forms, with or without
  modification, are permitted provided that the following conditions are met:
 Line 39 
 SUCH DAMAGE.
  -->
  <a href="./">Back to the index</a>
  <p><br>
- <h2>Misc.</h2>
+ <h2>Miscellaneous</h2>
  <p>
  <ul>
-   <li><a href="#networking">Networking</a>
+   <li><a href="#devel">Writing operating system code, or
-   <li><a href="#portmips">Porting operating systems to MIPS using GXemul</a>
+         developing firmware, using GXemul</a>
    <li><a href="#compilercontruct">Using GXemul in compiler contruction courses</a>
    <li><a href="#disk">How to start the emulator with a disk image</a>
+   <li><a href="#tape_images">How to start the emulator with tape images</a>
+   <li><a href="#disk_overlays">How to use disk image overlays</a>
+   <li><a href="#filexfer">Transfering files to/from the guest OS</a>
    <li><a href="#largeimages">How to extract large gzipped disk images</a>
    <li><a href="#userland">Running userland binaries</a>
    <li><a href="#promdump">Using a PROM dump from a real machine</a>
-Line 61 
 SUCH DAMAGE.
+Line 65 
 SUCH DAMAGE.
- <p><br>
- <a name="networking"></a>
- <h3>Networking:</h3>
- It is possible to let the guest OS running inside the emulator get access to
- the Internet. If you are interested in the technical details, and the
- reasons why networking is implemented in the emulator the way it currently
- is implemented, you might want to read the <a href="technical.html#net">
- networking section in the technical documentation</a>.
- <p>
- The guest OS running inside the emulator uses a private IPv4 address, such
- as 10.0.0.1, and the emulator acts as a NAT-like gateway/firewall at IPv4
- address 10.0.0.254. To the outside world it will seem like it is the host's
- OS that connects to other machines on the internet, not the guest OS.
- <p>
- <font color="#ff0000">NOTE: This is still experimental!
- As of 2004-07-21, ARP + ICMP + UDP + TCP are emulated well enough to let
- NetBSD and OpenBSD install via ftp, and use the network for many normal
- activities, but not everything works yet.</font>
  <p><br>
- <a name="portmips"></a>
+ <a name="devel"></a>
- <h3>Porting operating systems to MIPS using GXemul:</h3>
+ <h3>Writing operating system code, or developing firmware, using GXemul:</h3>
- Is this a good idea?  The answer is yes and no, depending on what you are
+ Is this a good idea?  The answer is yes and no, depending on the level of
- trying to port to. If you are developing an operating system or operating
+ detail you need in your simulations. If you are developing an operating
- system kernel of your own, and wish to target MIPS-like systems in general,
+ system or operating system kernel of your own, then the emulator can be a
- then the answer might be yes, for experimental purposes.
+ complement to testing on real hardware.
- <p>
+ <p>Important things to keep in mind:
- However, if you think that you can port an operating system
- to, say, the Silicon Graphics machine mode of GXemul and hope that your
- operating system will run on a real SGI machine, then you will most
- likely fail. GXemul simply does not emulate things well enough for that to work.
- Another example would be specific CPU details; if your code depends on,
- say, R10000 specifics, chances are that GXemul will not be sufficient.
- <p>
+ <ul>
- In many cases, hardware devices in GXemul are only implemented well
+         <li>Porting code to a specific machine mode, e.g. a Silicon Graphics
- enough to fool eg. NetBSD that they are working correctly, while in
+         machine, using GXemul, will not "magically" cause the code to
- fact they don't work very much at all.  Please keep this in mind, if you plan
+         work on a real machine. Sometimes code works in GXemul which doesn't
- to use GXemul when porting your code to MIPS.
+         work on real hardware, sometimes it's the other way around.
+         <p>
+         <li>GXemul contains bugs, and many things are not yet implemented.
+         <p>
+         <li><b>Very important!</b> I have only implemented devices in GXemul
+         to the degree that NetBSD, OpenBSD, Linux, etc don't complain too much.
+         <p>
+         If you are developing a driver for a device which is emulated by
+         GXemul, and your driver does not seem to be working, then the
+         probability of a bug in GXemul's implementation of the device is
+         very much higher than that of a bug in your driver.
+         <p>
+         The device implementations in GXemul are based on the assumption
+         that the emulated OS is already developed and bug-free. They are
+         not primarily intended to be used for development of new device
+         driver code in operating systems, so if you do that, then be
+         prepared for bugs and inconsitencies.
+         <p>
+         <li>CPU details in GXemul are usually wrong. If your code depends
+         on, say, R10000 or MIPS64 specifics, chances are that GXemul will
+         not be sufficient. One example is different revisions of ISAs;
+         some instructions which should trigger an exception on a
+         real MIPS processor usually execute anyway in GXemul. Another
+         example is if userland code tries to access kernel memory; in some
+         cases there is protection against this, but not in all cases (to get
+         higher performance).
+         <p>
+         <li>Caches. There is no cache emulation in GXemul right now. Caches
+         for R2000/R3000 are faked well enough to run NetBSD, Ultrix, etc
+         in the DECstation emulation mode, but other than that, cache
+         operations are treated as nops.
+ </ul>
+ <p>The bottom line is that GXemul can be useful as yet another way to test
+ your code during development, but it should not be fully relied on.
-Line 120 
 to use GXemul when porting your code to
+Line 130 
 to use GXemul when porting your code to
  <h3>Using GXemul in compiler contruction courses:</h3>
  If you are learning how to write a compiler, and wish to target a
- realistic target platform, then MIPS (as emulated by GXemul)
+ realistic target platform, then MIPS or ARM (as emulated by GXemul)
- might be a suitable choice.
+ might be suitable choices.
  <ul>
    <li><b>(+)</b>&nbsp;&nbsp;Your compiler needs to output real assembly
-         language code, which the assembler (eg gas, the GNU assembler) can
+         language code, which the assembler (e.g. gas, the GNU assembler) can
          then compile into object format, and then you need to link this
          into an executable image. This is much closer to how things work
          in real life than running assembly language listings in a simulator
-         (eg SPIM).
+         (e.g. SPIM).
    <p>
    <li><b>(-)</b>&nbsp;&nbsp;GXemul does not simulate out-of-order
          execution, penalties related to instruction scheduling, or
-Line 155 
 to get a list of possible options.
+Line 165 
 to get a list of possible options.
  Here are some examples. If you want to run a NetBSD/pmax kernel on an
  emulated DECstation machine, you would use a command line such as this:
  <pre>
-         $ <b>gxemul -E dec -e 3max -b -d pmax_diskimage.fs netbsd-pmax-INSTALL</b>
+         $ <b>gxemul -e 3max -d pmax_diskimage.fs netbsd-pmax-INSTALL</b>
  </pre>
- <p>
- NOTE: For some emulation modes, such as the DECstation mode, you do
+ <p>NOTE: For some emulation modes, such as the DECstation mode, you do
- <i>not</i> have to specify the name of the kernel, if the disk image is
+ <i>not</i> actually have to specify the name of the kernel, if the disk
- bootable!
+ image is bootable!
- <p>
- It is possible to have more than one disk. For each -d argument, a disk
+ <p>It is possible to have more than one disk. For each -d argument, a disk
  image is added; the first will be SCSI target 0, the second will be target 1, and so on,
  unless you specify explicitly which ID number the devices should have.
  <pre>
-         $ <b>gxemul -E dec -e 3max -b -d disk0.raw -d disk1.raw -d 5:disk2.raw netbsd-pmax-INSTALL</b>
+         $ <b>gxemul -e 3max -d disk0.raw -d disk1.raw -d 5:disk2.raw netbsd-pmax-INSTALL</b>
  </pre>
  Note: In the example above, disk2.raw will get scsi id 5.
- <p>
- If a filename has a 'c' prefix, or ends with ".iso", then it is assumed to be
+ <p>If a filename has a 'c' prefix, or ends with ".iso", then it is assumed to be
  a CDROM device (this can be overridden with a 'd' prefix, to force a read/write disk).
  For example, the following command would start the emulator with two
  CDROM images, and one harddisk image:
  <pre>
-         $ <b>gxemul -E dec -e 3max -b -d image.iso -d disk0.img -d c:second_cdrom.img netbsd-pmax-INSTALL</b>
+         $ <b>gxemul -e 3max -d image.iso -d disk0.img -d c:second_cdrom.img netbsd-pmax-INSTALL</b>
  </pre>
  Usually, the device with the lowest id becomes the boot device. To override
  this, add a 'b' prefix to one of the devices:
  <pre>
-         $ <b>gxemul -E dec -e 3max -b -d rootdisk.img -d bc:install-cd.iso name_of_kernel</b>
+         $ <b>gxemul -e 3max -d rootdisk.img -d bc:install-cd.iso name_of_kernel</b>
  </pre>
  If you have a physical CD-ROM drive on the host machine, say /dev/cd0c, you can
  use it as a CD-ROM directly accessible from within the emulator:
  <pre>
-         $ <b>gxemul -E dec -e 3max -b -d rootdisk.img -d bc:/dev/cd0c name_of_kernel</b>
+         $ <b>gxemul -e 3max -d rootdisk.img -d bc:/dev/cd0c name_of_kernel</b>
  </pre>
  It is probably possible to use harddisks as well this way, but I would not
  recommend it.
- <p>
+ <p><br>
+ <a name="tape_images"></a>
+ <h3>How to start the emulator with tape images:</h3>
  Using emulated tape drives is a bit more complicated than disks, because a
  tape can be made up of several "files" with space in between. The solution
  I have choosen is to have one file in the host's file system space for each
-Line 239 
 every time a tape is rewound, it is reop
+Line 258 
 every time a tape is rewound, it is reop
  on the command line. By changing what the symbolic name points to,
  you can "switch tapes" without quiting and restarting the emulator.
+ <p>
+ <font color="#ff0000">Note: Tape support is most likely very buggy,
+ because it has not been tested much, and has probably also suffered
+ from bit-rot by now.</font>
+ <p><br>
+ <a name="disk_overlays"></a>
+ <h3>How to use disk image overlays:</h3>
+ This is most likely best understood by an example:
+ <p><ul>
+   <li>Install e.g. <a href="guestoses.html#netbsdcatsinstall">NetBSD/cats</a>.
+         You will end up with a disk image called
+         <tt>nbsd_cats.img</tt>.
+   <p>
+   <li>Running the following command will boot straight from the disk
+         image, with no overlay images:<pre>
+         <b>gxemul -XEcats -d nbsd_cats.img netbsd.aout-GENERIC.gz</b>
+ </pre>
+   <li>You may now create an overlay file, a corresponding map file,
+         and start the emulator with the overlay image connected to
+         the same (explicit) ID as the base disk image:<pre>
+         <b>touch overlay.img overlay.img.map
+         gxemul -XEcats -d 0:nbsd_cats.img -d V0:overlay.img netbsd.aout-GENERIC.gz</b>
+ </pre>
+   <li>Any changes to the filesystem you perform when using the overlay
+         will only be written to that overlay. For example, to perform
+         a "roll back", you can do the following:<pre>
+         <b>rm -f overlay.img overlay.img.map
+         touch overlay.img overlay.img.map</b>
+ </pre>
+         and then simply start the emulator again, with the newly created
+         overlay image.
+ </ul>
+ <p>It is also possible to add multiple overlays. In that case, writes
+ always go the the <i>last</i> added overlay.
+ <p>GXemul uses Unix' way of supporting files with "holes",
+ so even if <tt>ls -l overlay.img</tt> says that the overlay is several
+ gigabytes large, <tt>du overlay.img</tt> should reveal that only the
+ blocks that have actually been written to have been stored in the
+ overlay, e.g.:<pre>
+         <b>$ ls -l
+         ..
+         -rw-r--r--  1 debug  wheel  3072319488 Mar 24 11:59 nbsd_cats.img
+         -rw-r--r--  1 debug  wheel     2465354 Mar 24 11:44 netbsd.aout-GENERIC.gz
+         -rw-r--r--  1 debug  wheel  2930841600 Mar 24 14:02 overlay.img
+         -rw-r--r--  1 debug  wheel      715538 Mar 24 14:02 overlay.img.map
+         $ du overlay.img
+    overlay.img
+ </b>
+ </pre>
+ <p>The .map file is simply a raw bitmap telling which blocks of the
+ overlay file that are in use.
+ <p><br>
+ <a name="filexfer"></a>
+ <h3>Transfering files to/from the guest OS:</h3>
+ If the emulated machine supports networking (see <a
+ href="networking.html#intro">this section</a> for more info), then the easiest
+ way to transfer files is probably to use FTP or similar methods.
+ <p>There is another way of transfering files which works for any kind of
+ emulated machine which supports disks (either SCSI or IDE). Any file can
+ be supplied as a disk image. For example, consider the following:<pre>
+         $ <b>gxemul -XEcats -d nbsd_cats.img -d archive.tar.gz netbsd-GENERIC</b>
+ </pre>
+ This will start NetBSD/cats with <tt>nbsd_cats.img</tt> as IDE master on
+ controller 0 (wd0), and <tt>archive.tar.gz</tt> as IDE slave on controller
+(wd1). From inside NetBSD, it is now possible to extract the files using
+ the following command:<pre>
+         (inside emulated NetBSD/cats)
+         # <b>tar zxvf /dev/wd1c</b>
+ </pre>
+ Don't worry if NetBSD complains about lack of disklabel; it doesn't
+ matter. On some machines, NetBSD uses <tt>wd1d</tt> instead of
+ <tt>wd1c</tt> for the entire disk.
+ There is also a minor problem: reading the end of the disk image. If you
+ experience problems untaring archives like this, then pad out the archive
+ first with some zeroes.
+ <p>Transfering files <i>out</i> from the emulated operating system to the
+ host can be done the same way. First, prepare an empty archive file:<pre>
+         $ <b>dd if=/dev/zero of=newarchive.tar bs=1024 count=1 seek=10000</b>
+ </pre>This example created a 10 MB empty file. Then, start the emulator
+ like this:<pre>
+         $ <b>gxemul -XEcats -d nbsd_cats.img -d archive.tar netbsd-GENERIC</b>
+ </pre>
+ and transfer files by creating an archive directly onto the disk image:<pre>
+         (inside emulated NetBSD/cats)
+         # <b>tar cvf /dev/wd1c filenames</b>
+ </pre>
+ where filenames are the files or directories to transfer.
  <p><br>
-Line 285 
 count.)
+Line 416 
 count.)
  <a name="userland"></a>
  <h3>Running userland binaries:</h3>
- You can run (some) userland programs as well. This will not emulate any
+ <font color="#ff0000"><b>NOTE 2007-06-15:</b> I just removed most of the
- particular machine, but instead try to translate syscalls from for example
+ userland (syscall) emulation support, and started a rewrite from scratch.
- NetBSD/pmax into the host's OS' syscalls. Right now, this is just a
+ The rest of this section in the documentation is not currently valid.</font>
- proof-of-concept, to show that it would work; there's lots of work left to
- do to make it actually run useful programs (for example dynamically linked
+ <p>There is some skeleton code for running userland programs as well. This
- programs).
+ will not emulate any particular machine, but instead try to translate
+ syscalls from e.g. NetBSD/pmax into the host's OS' syscalls. Right now,
+ this is just a proof-of-concept, to show that it could work; there's lots
+ of work left to do to make it actually run useful programs.
  <p>
-Line 321 
 programs).
+Line 455 
 programs).
          tab.csbnet.se
  </pre>
+ <!--
    <p>
    <li><b>NetBSD/powerpc:</b>
          <br>
-Line 344 
 programs).
+Line 479 
 programs).
          Hello, world!
  </pre>
+ -->
  </ul>
-Line 411 
 and store in a file. Let's call this fil
+Line 547 
 and store in a file. Let's call this fil
  </pre>
  This binary image can now be used in the emulator:
  <pre>
-         $ <b>gxemul -E dec -e 3min -Q -M128 -q 0xbfc00000:DECstation5000_125_promdump.bin</b>
+         $ <b>gxemul -e 3min -Q -M128 -q 0xbfc00000:DECstation5000_125_promdump.bin</b>
          KN02-BA V5.7e
          ?TFL:  3/scc/access (1:Ln1 reg-12: actual=0x00 xpctd=0x01) [KN02-BA]
-Line 452 
 This binary image can now be used in the
+Line 588 
 This binary image can now be used in the
           osconsole=3
          >>
  </pre>
- <i>(Note: at the moment, this doesn't work. I must have broken something when
- fixing something else, but this is what it looked like at the time.)</i>
+ <p><font color="#ff0000">(Note: at the moment, this doesn't work.
- <p>
+ I must have broken something when fixing something else, but this
- During bootup, the PROM complains <i>a lot</i> about hardware failures.
+ is what it looked like at the time.)</font>
+ <p>During bootup, the PROM complains <i>a lot</i> about hardware failures.
  That's because the emulator doesn't emulate the hardware well enough yet.
- <p>
- The command line options used are: -E dec for DECstation, -e 3min for
+ <p>The command line options used are: <tt>-e 3min</tt> for
- "model 3" (5000/1xx), -Q to supress the emulator's own PROM
+ "DECstation 3min" (5000/1xx), <tt>-Q</tt> to supress the emulator's own PROM
- call emulation, -M128 for 128MB RAM (because GXemul doesn't correctly
+ call emulation, <tt>-M128</tt> for 128MB RAM (because GXemul doesn't correctly
  emulate memory detection well enough for the PROM to accept, so it will
- always believe there is 128MB ram anyway), and -q to supress debug messages.
+ always believe there is 128MB ram anyway), and <tt>-q</tt> to supress debug messages.
- The 0xbfc00000 in front of the filename tells GXemul that it is a raw
+ The <tt>0xbfc00000</tt> in front of the filename tells GXemul that it is a raw
  binary file which should be loaded at a specific virtual address.
  <p><br>
- <h4>Dumping the PROM on a SGI O2:</h4>
+ <a name="promdump_o2"><h4>Dumping the PROM on a SGI O2:</h4></a>
  The general ideas in this section applies to using ROM images from other
- machines as well. Besides DECstation, I've also tried this on an SGI IP32
+ machines as well. I have also tried this on an SGI IP32 ("O2"), in addition
- ("O2").
+ to the DECstation.
- <p>
- For the O2, a suitable command to dump the prom memory range is
+ <p>For the O2, a suitable command to dump the prom memory range is
  <pre>
          &gt;&gt; <b>dump -b 0xBFC00000:0xBFC80000</b>
  </pre>
  Make sure you capture all the output (via serial console) into a file,
- and then run experiments/sgiprom_to_bin on the captured file.
+ and then run <tt>experiments/sgiprom_to_bin</tt> on the captured file.
- <p>
- (2005-01-16: The emulator doesn't really emulate the IP32 well enough to
- actually run the PROM image without using special hacks, but it might do
- so some time in the future.)
+ <p>
+ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
+ <a href="sgi-o2-real.jpg"><img src="sgi-o2-real_small.jpg"></a>
+ &nbsp;&nbsp;&nbsp;
+ <a href="20050817-sgi-o2-success-7.png"><img src="20050817-sgi-o2-success-7_small.png"></a>
+ &nbsp;&nbsp;&nbsp;
+ <a href="20050817-sgi-o2-success-8.png"><img src="20050817-sgi-o2-success-8_small.png"></a>
+ <p>The photo on the left is from the real machine. The other two are
+ screenshots of the PROM running experimentally in GXemul, using <tt>-Y2</tt>
+ framebuffer scaledown.
+ <p>Normally during bootup, the IP32 PROM does a Power-On test which makes
+ sure that the caches and other things are working properly. GXemul doesn't
+ emulate all those things well enough for the tests to pass. The
+ experimental screenshots above were taken with cache detection skipped
+ manually.
+ <p><font color="#ff0000">
+ In other words: don't expect this to work out-of-the-box with GXemul right
+ now. It might work once I've added correct cache emulation.</font>
+ <p>The command line used to start the emulator, once correct cache
+ emulation has been implemented, would be something like <tt>gxemul -XQeo2
+xbfc00000:prom.bin</tt>.
+ <p>The same caution applies when dealing with SGI PROMs as with
+ DECstation PROMs: GXemul doesn't really emulate the hardware, it only
+ "fakes" devices well enough to fool some things, primarily NetBSD, that
+ it is emulating a real machine. ROM code is usually a <i>lot</i> more
+ picky about the details.
+ <p>The graphics used in the O2 is (as far as I know) undocumented. Combining
+ some traces of info from how Linux/O2 draws to the screen with some
+ reverse-engineering of my own, I've implemented enough of the controller to
+ let the PROM draw rectangles and bitmaps, but not much more. The SCSI
+ controller is not implemented yet either.

 Legend:



Removed from v.8
 


changed lines


 
Added in v.42
 Legend:



Removed from v.8
 


changed lines


 
Added in v.42
-Removed from v.8
+Added in v.42

	ViewVC Help
Powered by ViewVC 1.1.26