4-Remotely Controlling(远程控制) an Octeon through PCI, PCIe, or EJTAG

1. Introduction

In development, and production, it may be useful to access and control Octeon from a remote host or other communication path. For example, a PC may need to boot Octeon through PCI/PCIe. At other times you may want to read registers through EJTAG. In order to support these conditions, Cavium Networks has developed a remote protocol library and a number of special purpose command line utilities. These utilities allow common operations like register dumps, read and write of CSRs, and profiling to be done outside of Octeon.

This document provides detailed information on the following aspects of controlling Octeon remotely:

• 1. Introduction
• 2. Supported Remote Protocols
• 3. Backwards Compatibility with Previous Utilities
• 4. Booting over PCI or EJTAG (oct-remote-boot)
• 5. Remote Bootloader Commands (oct-remote-bootcmd)
• 6. Remote Console (oct-remote-console)
• 7. Read CPU Core State (oct-remote-core)
• 8. CSR Access (oct-remote-csr)
• 9. Quickly Test Memory (oct-remote-ddr)
• 10. Loading Files into Memory (oct-remote-load)
• 11. Read / Write a 64 bit Memory Location (oct-remote-memory)
• 12. Displaying Octeon POW state (oct-remote-pow)
• 13. Profiling (oct-profile, oct-remote-profile)
• 14. Resetting Octeon (oct-remote-reset)
• 15. Saving Memory to a file (oct-remote-save)
• 16. Tracing using the Octeon Trace Buffer (oct-remote-tra)
• 17. Determining proper DDR configuration parameters (oct-remote-ddr-tune)
• 18. Debugging using GDB

2. Supported Remote Protocols

The Octeon remote library supports a number of possible protocols for connecting to Octeon. The choice of which protocol to use is controlled by the environment variableOCTEON_REMOTE_PROTOCOL. If this environment variable isn't set, the utilities will display a message showing the possible supported protocols. Here is a list of some of the support protocols:

• OCTEON_REMOTE_PROTOCOL = PCI[:device]
  • Use PCI/PCIe to access Octeon from the host or root complex. If multiple Octeon's are present, the optional "device" number specifies which one to use. The first device is number 0.
• OCTEON_REMOTE_PROTOCOL = PCIHOST
  • Use PCI/PCIe to connect to the host Octeon from a target. This is normally only used during development for profiling and register dumps. This allows an Octeon PCI/PCIe target to access the host's (or root complex's) registers.
• OCTEON_REMOTE_PROTOCOL = MACRAIGOR:name,tcp_port
  • Use a Macraigor mpDemon to access Octeon through EJTAG. "name" is either the host name or IP address of the mpDemon probe. "tcp_port" is the TCP port of the probe (normally 1000).
• OCTEON_REMOTE_PROTOCOL = GDB:name,tcp_port
  • Use the GDB remote protocol to access Octeon. Many brands of EJTAG probes supply GDB remote stubs for debugging. The remote utilities can use this protocol directly. Abatron, Mentor Graphics, and Macraigor are all known to work with this protocol. "name" is either the host name or IP address of the GDB stub. "tcp_port" is the TCP port of the first core. Each additional core increments the TCP port by one. The README.txt files in the sdk/ejtag directory have more information about specific probes.
• OCTEON_REMOTE_PROTOCOL = LINUX
  • Assume the utility is running under Linux userspace native on the Octeon. mmap() and debug exception handlers are used to access the local Octeon without modifying the kernel.

In the event of problems, the environment variable OCTEON_REMOTE_DEBUG can be used to enable detailed diagnostic information. Increasing numbers, show more detail:

• OCTEON_REMOTE_DEBUG = 0
  • Disables all diagnostic output.
• OCTEON_REMOTE_DEBUG = 1
  • Enables a small amount of informational data.
• OCTEON_REMOTE_DEBUG = 2
  • More diagnostic data is generated.
• OCTEON_REMOTE_DEBUG = 3
  • Call traces of octeon-remote are generated.
• OCTEON_REMOTE_DEBUG = ...
• OCTEON_REMOTE_DEBUG = 9
  • All possible output is enabled.

Normally debug output from the remote library is color coded based on the diagnostic level. If you wish to turn the color off, add 256 to the debug level.

3. Backwards Compatibility with Previous Utilities

The Octeon SDK contains a number of trivial shell scripts to allow the oct-remote-* utilities to act as a drop in replacement for the oct-pci-* and oct-linux-* utilities provided in previous SDKs. The functionality of the older utilities is implemented in the remote utilities when the OCTEON_REMOTE_PROTOCOL is set appropriately.

• The scripts for oct-pci-* set OCTEON_REMOTE_PROTOCOL=pci.
• The scripts for oct-linux-* set OCTEON_REMOTE_PROTOCOL=linux.

4. Booting over PCI or EJTAG (oct-remote-boot)

Octeon can be booted from a remote host without the need for a boot flash on the Octeon board. For PCI targets, the Octeon board must be configured for PCI boot mode. This keeps all cores in reset after the chip is taken out of reset. For EJTAG boot, the board must _not_ be configured for PCI boot mode, as core 0 must be running for the EJTAG probe to use. The oct-remote-boot program configures the memory controller, copies the bootloader to DRAM, and then boots the bootloader. By default the provided bootloader (target/bin/u-boot-pci.bin) for the detected board is used. A different bootloader image can be specified on the command line. oct-remote-boot also supports a 'generic' board type and bootloader that should support most Octeon boards. In order to use the generic board mode, certain board configuration parameters must be passed on the command line so that oct-remote-boot can configure the memory controller properly. (The SPD addresses for the DIMMS are required in all cases.)

Usage: ./oct-remote-boot [--memonly] [--board=board_name] [bootloader filename]    Boot Octeon and load bootloader over remote connection (PCI or EJTAG).    --help: Show this usage.    --memonly: If specified only the memory controller is configured.    --board=board_name:  Force a particular board type to be used.    --envfile=<environment filename>: Reads the u-boot environment data from              the specified file, loads it into DRAM for u-boot to use.  The u-boot              image must be configured to use this environment.  The format of the file              is one environment variable per line:              env_var=env_val\n              each line must be ended with a single '\n'    --scantwsi: Scan twsi bus and exit.    --dumpspd: Dump SPD contents when scanning twsi bus.    The following options are to support booting the generic ram based u-boot image, and    are not used for normal operation.    --ddr0spd=<twsi_addr[,twsi_addr]|filename[,filename]>: SPD addresses or files for DIMMS on interface 0.  Specify multiple times               if multiple dimms are present.  For 128 bit wide DDR interaces (CN38XX/CN58XX)               comma separated pairs off addresses must be used.(only required for generic board type.)    --ddr1spd=<twsi_addr|filename>: SPD addresses or files for DIMMS on DDR interface 1.  Only valid for Chips               with 2 DDR controllers.  Specify multiple times for multiple DIMMS.(only required for generic board type.)    --ddr0spd=<twsi addrs>: SPD addresses for DIMMS on interface 0.  Specify multiple times               if multiple dimms are present.  For 128 bit wide DDR interaces (CN38XX/CN58XX)               comma separated pairs off addresses must be used. (only required for generic board type.)    --ddr1spd=<twsi addrs>: SPD addresses for DIMMS on DDR interface 1.  Only valid for Chips               with 2 DDR controllers, such as CN56XX.  Specify multiple times for multiple DIMMS.    --cpu_ref_hz=<NNN>: CPU reference clock in Hz.  (only requried for generic board type.)    --ddr_ref_hz=<NNN>: DDR reference clock in Hz.  (only requried for generic board type.)    --ddr_clock_hz=<NNN>: DDR clock in Hz.  (only requried for generic board type.)    --board_delay=<NNN>: board delay, in picoseconds. (only required for generic board type.)    --readlevel:  Enable read leveling (auto board delay), CN52XX/CN56XX only.    The bootloader (optionally) supplied must be built for     booting from DRAM.  If no bootloader is specified the standard one     for the detected or supplied board type will be chosen.For boards without SPD eeproms, filenames rather than SPD addresses may be specified.  The filesmust contain SPD values with 1 hex value per line, and a minimum of 64 values.

Usage examples:

    $ sudo oct-remote-boot    All cores in reset, skipping soft reset.    Using bootloader image: ..../sdk/target/bin/u-boot-octeon_thunder_pciboot.bin    Initialized 2048 MBytes of DRAM    $

    $ sudo oct-remote-boot my-bootloader-image.bin    Found Octeon on bus 0 in slot 5. BAR0=0xf8000000, BAR1=0xf0000000    $

    $ sudo oct-remote-boot --scantwsi    Scanning twsi bus...    Twsi addr 81 (0x51) responded, DIMM SPD    Twsi addr 82 (0x52) responded, DIMM SPD    Twsi addr 83 (0x53) responded    Twsi addr 104 (0x68) responded    $

    $ sudo oct-remote-boot --ddr0spd=0x51,0x52 --board_delay=5000 --board=generic    Using user supplied DDR 0 spd address(es): 0x51,0x52    Using user supplied board delay: 5000    Using user supplied board name: generic, number: 28    WARNING: detected board type THUNDER (6) does no match supplied board type GENERIC (28).    NOTICE: using default 50000000 Hz DDR reference clock.    NOTICE: using default 50000000 Hz CPU reference clock.    Using bootloader image: ..../sdk/target/bin/u-boot-octeon_generic_ram.bin    Initialized 2048 MBytes of DRAM    $

5. Remote Bootloader Commands (oct-remote-bootcmd)

The bootloader can accept commands remotely. Note that the command must be quoted if it contains whitespace. Also, any u-boot environment variables that are referenced must be escaped so that the shell does not expand them.

    $ sudo oct-remote-bootcmd "bootoct \$(fileaddr) coremask=ffff"    Found Octeon on bus 0 in slot 5. BAR0=0xf8000000, BAR1=0xf0000000    $

6. Remote Console (oct-remote-console)

The bootloader, simple executive applications, and Linux can direct their consoles over remote protocols to the oct-remote-console utility. This allows interaction with programs running on Octeon without a serial connection. Multiple separate consoles can be used, so for example each core could have its own console if it is running linux.

Note:

Applications that do not link against this simple executive library cannot use the remote console.

U-boot, and many of the CVMX APIs refer to remote consoles support as "PCI Console". Historically, PCI was the first protocol supported. Later PCIe and EJTAG were added.

Bootloader usage:

The console data structures are initialized by the bootloader at boot time if configured. If the bootloader does not initialize the data structures, console functionality will not be available. The 'pci_console_count' and 'pci_console_size' are used to control the number of consoles and the size of each console's buffers. By default the console structures are not initialized. Setting the 'pci_console_count' to a non-zero value will cause the data structures to be initialized. If this variable is saved in the environment, then the data structures will be initialized at boot time as well. The console data structures will be configured, but the remote console will not be used by the bootloader unless the 'pci_console_active' environment variable is set. If only the 'pci_console_active' variable is set, then the default of 1 console with 1024 byte buffers will be used. The bootloader always uses remote console 0, so the following command (run as root) will connect to it:

    $ oct-remote-console 0    Found Octeon on bus 3 in slot 2. BAR0=0xe7fff000, BAR1=0xd8000000    Connecting to PCI console 0    Using raw terminal mode, escape character is ^A, use ^A D  to exit, ^A A to send ^A    $

Simple Executive usage:

If the bootloader is configured to use the remote console, the simple executive applications will also use console 0.

Linux usage:

Linux uses a command line argument to select a remote console. The argument 'console=pciN' should be added to the command line, with N being replaced with the number of the console to be used.

oct-remote-console utility usage:

    Usage: oct-remote-console [--noraw] <CONSOLE_NUMBER>

The oct-remote-console utility takes the console number to connect to as an argument. By default it put the terminal in raw mode so that control characters are passed through to Octeon. The escape character is ^A, and to exit type ^A D. The utility can also be invoked with the '--noraw' mode in which case control characters like ^C and ^Z are not passed through, but cause signals to be sent to the oct-remote-console utility. In 'noraw' mode, ^C causes the utility to exit.

Note:

Because the oct-remote-console utility polls Octeon, if Octeon is reset while the console is connected there is a good chance that the entire machine will lock up. (The lockup is due to Octeon being non-responsive to the PCI accesses while it is in reset.) The oct-remote-console utility should be exited before reseting the Octeon. The one exception to this is that oct-remote-reset and oct-remote-boot can be used while the pci console is active since they coordinate with oct-remote-console to ensure that no accesses happen while the Octeon is non-responsive.

7. Read CPU Core State (oct-remote-core)

The general purpose registers, coprocessor 0 state, and TLB can be read remotely using oct-remote-core. A single argument specifies the Octeon core number to read.

Note:

Some registers are not available through some remote protocols. These registers will show as 0x0bad0bad0bad0bad.

Usage: ./oct-remote-core <core>    Display the state of an Octeon core.    core - Octeon core to display.

Sample output from core 0 in U-boot

Core 0CPU registers:   zero (r0)    0x0000000000000000         s0 (r0)      0x0000000000000001     at (r1)    0x0000000000000000         s1 (r1)      0xffffffff800bfa48     v0 (r2)    0x0000000000000000         s2 (r2)      0xffffffff800a50a8     v1 (r3)    0x8001180000000828         s3 (r3)      0x0000000000000010     a0 (r4)    0x0000000000000001         s4 (r4)      0xffffffff800a0000     a1 (r5)    0xffffffff800b06e0         s5 (r5)      0xffffffff8005ff24     a2 (r6)    0x0000000000000000         s6 (r6)      0x0000000000024108     a3 (r7)    0xffffffff800b06e0         s7 (r7)      0x6f513832dea27065     a4 (r8)    0x0000000000000000         t8 (r8)      0x0000000000000000     a5 (r9)    0xffffffffffffffd0         t9 (r9)      0xffffffff8008542c     a6 (r10)   0x0000000000000002         k0 (r10)     0xffffffff8005ff24     a7 (r11)   0xffffffff800b05a0         k1 (r11)     0xfffffffffffffffe     t0 (r12)   0xffffffff800b06e0         gp (r12)     0xffffffff800b06e0     t1 (r13)   0xffffffff800ad3b0         sp (r13)     0xffffffff8005bd10     t2 (r14)   0x0000000000000020      s8/fp (r14)     0xa89c196f513832de     t3 (r15)   0xffffffff800b06e0         ra (r15)     0xffffffff80075584     lo         0x0000000000000ea6         hi           0x0000000000000000Coprocessor 0 registers:  ($ 0,0) Index     0xffffffff80000000  ($18,1) WatchLo1       0x0bad0bad0bad0bad  ($ 1,0) Random    0x000000000000003f  ($19,0) WatchHi0       0xffffffff80000000  ($ 2,0) EntryLo0  0x0000000000000000  ($19,1) WatchHi1       0x0bad0bad0bad0bad  ($ 3,0) EntryLo1  0x0000000000000000  ($20,0) XContext       0x0000000000000000  ($ 4,0) Context   0x0000000000000000  ($23,0) Debug          0x000000004011fc20  ($ 5,0) PageMask  0x0000000000000000  ($25,0) PerfControl0   0xffffffffc0000000  ($ 5,1) PageGrain 0x0000000020000000  ($25,2) PerfControl1   0x0000000040000000  ($ 6,0) Wired     0x0000000000000000  ($24,0) DEPC           0xffffffff80075560  ($ 7,0) HWREna    0xffffffffc000000f  ($25,1) PerfCount0     0x0000000000000000  ($ 8,0) BadVAddr  0x0000000000000000  ($25,3) PerfCount0     0x0000000000000000  ($ 9,0) Count     0xffffffffee38d00c  ($27,0) IcacheError    0x007f7f0000001850  ($10,0) EntryHi   0xc001ffff9007e000  ($27,1) DcacheError    0x00000000000019d0  ($11,0) Compare   0x0000000000000000  ($28,0) IcacheTagLo    0xcff1ffffe1f81281  ($12,0) Status    0x00000000504000e5  ($28,1) IcacheDataLo   0x5f7c42f1cb820613  ($12,1) IntCtl    0xfffffffffc000000  ($28,2) DcacheTagLo    0x0000000000000000  ($12,2) SRSCtl    0x0000000000000000  ($28,3) DcacheDataLo   0x0000000000000060  ($13,0) Cause     0x0000000040008000  ($29,1) IcacheDataHi   0x0000000000000000  ($14,0) EPC       0x27178f42b6d265de  ($29,2) DcacheTagHi    0x0000000000000000  ($15,0) PRId      0x00000000000d0708  ($29,3) DcacheDataHi   0x0000000000000000  ($15,1) Ebase     0xffffffff80000000  ($30,0) ErrorEPC       0xffffffffff200e3c  ($16,0) Config    0xffffffff8000c48a  ($31,0) DESAVE         0x0000000000000000  ($16,1) Config1   0xfffffffffe3303da  ($ 9,7) CvmCtl         0x00000000b00043f0  ($16,2) Config2   0xffffffff80000367  ($11,7) CvmMemCt       0x0000000000046104  ($16,3) Config3   0x0000000000001080  ($ 9,6) CvmCount       0x000000015d5c0520  ($18,0) WatchLo0  0x0000000000000000  ($22,0) MultiCoreDebug 0x0bad0bad0bad0badTLB: 0: Virtual=0xffffffff9007e000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB 1: Virtual=0xffffffff9007c000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB 2: Virtual=0xffffffff9007a000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB 3: Virtual=0xffffffff90078000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB 4: Virtual=0xffffffff90076000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB 5: Virtual=0xffffffff90074000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB 6: Virtual=0xffffffff90072000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB 7: Virtual=0xffffffff90070000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB 8: Virtual=0xffffffff9006e000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB 9: Virtual=0xffffffff9006c000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB10: Virtual=0xffffffff9006a000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB11: Virtual=0xffffffff90068000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB12: Virtual=0xffffffff90066000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB13: Virtual=0xffffffff90064000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB14: Virtual=0xffffffff90062000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB15: Virtual=0xffffffff90060000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB16: Virtual=0xffffffff9005e000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB17: Virtual=0xffffffff9005c000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB18: Virtual=0xffffffff9005a000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB19: Virtual=0xffffffff90058000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB20: Virtual=0xffffffff90056000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB21: Virtual=0xffffffff90054000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB22: Virtual=0xffffffff90052000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB23: Virtual=0xffffffff90050000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB24: Virtual=0xffffffff9004e000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB25: Virtual=0xffffffff9004c000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB26: Virtual=0xffffffff9004a000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB27: Virtual=0xffffffff90048000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB28: Virtual=0xffffffff90046000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB29: Virtual=0xffffffff90044000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB30: Virtual=0xffffffff90042000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB31: Virtual=0xffffffff90040000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB32: Virtual=0xffffffff9003e000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB33: Virtual=0xffffffff9003c000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB34: Virtual=0xffffffff9003a000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB35: Virtual=0xffffffff90038000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB36: Virtual=0xffffffff90036000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB37: Virtual=0xffffffff90034000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB38: Virtual=0xffffffff90032000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB39: Virtual=0xffffffff90030000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB40: Virtual=0xffffffff9002e000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB41: Virtual=0xffffffff9002c000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB42: Virtual=0xffffffff9002a000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB43: Virtual=0xffffffff90028000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB44: Virtual=0xffffffff90026000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB45: Virtual=0xffffffff90024000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB46: Virtual=0xffffffff90022000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB47: Virtual=0xffffffff90020000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB48: Virtual=0xffffffff9001e000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB49: Virtual=0xffffffff9001c000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB50: Virtual=0xffffffff9001a000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB51: Virtual=0xffffffff90018000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB52: Virtual=0xffffffff90016000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB53: Virtual=0xffffffff90014000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB54: Virtual=0xffffffff90012000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB55: Virtual=0xffffffff90010000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB56: Virtual=0xffffffff9000e000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB57: Virtual=0xffffffff9000c000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB58: Virtual=0xffffffff9000a000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB59: Virtual=0xffffffff90008000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB60: Virtual=0xffffffff90006000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB61: Virtual=0xffffffff90004000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB62: Virtual=0xffffffff90002000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB63: Virtual=0xffffffff90000000 Page0=0x00000000,C=0,D=0,V=0,G=0 Page1=0x00000000,C=0,D=0,V=0,G=0 ASID=  0 Size=4KB

8. CSR Access (oct-remote-csr)

Octeon CSRs may be read or written directly from a remote host. The utility oct-remote-csr may be used to access any 64bit CSR available on Octeon.

Usage: ./oct-remote-csr CSR [value] [CSR [value] ...]    Read or write a CSR by name.    CSR     Name of CSR to read or write. Case is ignored.    value   If present, write this value to the CSR. Otherwise do a read.            The value can be in decimal of hex (0x...).    If CSR is a partial name, the list of CSRs starting with it are displayed.

Here is an example of reading the CIU enable register for Core 0 interrupt line 2:

$ ./oct-remote-csr CIU_INT0_EN0CIU_INT0_EN0(0x0001070000000200) = 0x4100410300000000  [   63] BOOTDMA              =          0 (0x0)  [   62] MII                  =          1 (0x1)  [   61] IPDPPTHR             =          0 (0x0)  [   60] POWIQ                =          0 (0x0)  [   59] TWSI2                =          0 (0x0)  [58:57] RESERVED_57_58       =          0 (0x0)  [   56] USB                  =          1 (0x1)  [55:52] TIMER                =          0 (0x0)  [   51] RESERVED_51_51       =          0 (0x0)  [   50] IPD_DRP              =          0 (0x0)  [   49] RESERVED_49_49       =          0 (0x0)  [   48] GMX_DRP              =          0 (0x0)  [   47] TRACE                =          0 (0x0)  [   46] RML                  =          1 (0x1)  [   45] TWSI                 =          0 (0x0)  [   44] RESERVED_44_44       =          0 (0x0)  [43:40] PCI_MSI              =          1 (0x1)  [39:36] PCI_INT              =          0 (0x0)  [35:34] UART                 =          0 (0x0)  [33:32] MBOX                 =          3 (0x3)  [31:16] GPIO                 =          0 (0x0)  [15: 0] WORKQ                =          0 (0x0)

9. Quickly Test Memory (oct-remote-ddr)

A very simple, and not thorough, memory test can be run remotely using oct-remote-ddr. This utility should only be used as a basic alive test. The hw-ddr2 diagnostics implements a much better memory test. The DRAM controller on Octeon must be already configured before this test can be run.

$ oct-remote-ddr -hOcteon DDR2 memory test over a remote interfaceNote: Requires use of oct-*-boot to configure memory controllerUsage:    ./oct-remote-ddr  <Mbytes to test>

10. Loading Files into Memory (oct-remote-load)

Any file may be loaded into Octeon memory using the oct-remote-load utility. This utility makes a great alternative to Kermit or the compact flash for loading programs. The example below loads the passthrough example into Octeon memory. The address of 0 causes the oct-remote-load utility to determine the address of memory reserved for loading by the bootloader and use that address for loading the file. A fixed (non 0) address may also be given.

    $ sudo oct-remote-load 0 passthrough    Found Octeon on bus 0 in slot 5. BAR0=0xf8000000, BAR1=0xf0000000    $

At this point you can start passthrough with the bootloader command:

    $ sudo oct-remote-bootcmd "bootoct 0 numcores=16"

11. Read / Write a 64 bit Memory Location (oct-remote-memory)

Read or write a single 64bit value from Octeon's memory.

$ ./oct-remote-memoryUsage: ./oct-remote-memory address [value]    Read or write 8 bytes at a  memory address.    address Address of memory to read or write.    value   If present, write this value to the address. Otherwise do a read.

12. Displaying Octeon POW state (oct-remote-pow)

The state of the Octeon POW can be captured and displayed through the remote interface. oct-remote-pow captures the state of all the input queues, de-schedue lists, no-schedule lists, the external memory queue pointers, and the complete state of each core. Without any arguments oct-remote-pow attempts to capture the state without disturbing the running system. This can cause the resulting capture state to be inconsistent due to changes while the capture was running. The -s,--stop-cores option tells the utility to temporarily stop the cores during the capture. By placing the Octeon cores into debug mode the captured state is normally consistent for the time required to make the capture. Here is a list of the important items shown in the state capture:

• The POW internal freelist head and tail pointers
• The tag state of each core including any pending tag switches and work requests. Each core also has the previous and next pointer for tags in ordered and atomic lists.
• The queue head and tail pointers for each group's no-schedule list.
• The queue head and tail pointers for each group's de-schedule list.
• The queue head and tail pointers for each of the 8 input queues.
• The queue head and tail pointers for each of the 8 input queue memory lists. Each input queue has two memory lists (a/b).
• The current tag type, tag, group, work queue pointer, and next link pointer for every internal POW entry. Also each entry is marked with the queue it is currently in. Entries on the no-schedule and de-schedule lists also contain pending tag switch information.

13. Profiling (oct-profile, oct-remote-profile)

Profiling of any code running on Octeon can be performed using the oct-profile and oct-remote-profile utilities. This program samples the core PC locations remotely for a very low impact statistical based profiler. Here is the basic usage information:

    Usage: oct-profile [--line] [--address] <elf>        <elf>        Mips ELF files to load debugging information from.        [--line]     Show line number information.        [--address]  Show exact PC addresses.        [--perf1=<Core counter>] Setup Core performance counter 1        [--perf2=<Core counter>] Setup Core performance counter 2        [--l2perf1=<L2 counter>] Setup L2 performance counter 1        [--l2perf2=<L2 counter>] Setup L2 performance counter 2        [--l2perf3=<L2 counter>] Setup L2 performance counter 3        [--l2perf4=<L2 counter>] Setup L2 performance counter 4    Possible Core counters:        none clk issue ret nissue sissue dissue ifi        br brmis j jmis replay iuna trap        uuload uustore uload ustore ec mc cc csrc        cfetch cpref ica ii ip cimiss        wbuf wdat wbufld wbuffl wbuftr badd baddl2 bfill        ddids idids didna lds lmlds iolds dmlds        sts lmsts iosts iobdma dtlb dtlbad itlb        sync synciob syncw    Possible L2 counters:        cycles imiss ihit dmiss        dhit miss hit victim-buffer-hit        lfb-nq-index-conflict tag-probe tag-update tag-probe-completed        tag-dirty-victim data-store-nop data-store-read data-store-write        memory-fill-data-valid memory-write-request memory-read-request memory-write-data-valid        xmc-nop xmc-ldt xmc-ldi xmc-ldd        xmc-stf xmc-stt xmc-stp xmc-stc        xmc-dwb xmc-pl2 xmc-psl1 xmc-iobld        xmc-iobst xmc-iobdma xmc-iobrsp xmd-bus-valid        xmd-bus-valid-dst-l2c xmd-bus-valid-dst-iob xmd-bus-valid-dst-pp rsc-nop        rsc-stdn rsc-fill rsc-refl rsc-stin        rsc-scin rsc-scfl rsc-scdn rsd-data-valid        rsd-data-valid-fill rsd-data-valid-strsp rsd-data-valid-refl lrf-req        dt-rd-alloc dt-wr-inva

The profile can be used with any ELF file with DWARF debugging. It may be used for the simple executive, or Linux userspace applications. Here is an example of profiling passthrough.

    $ sudo oct-profile passthrough      L2:         0 Icache Misses,   8929700 Dcache Misses,  72780967 Dcache Hits,  89707898 Writes      40    2    2    2    2    2    1    7    4    8    4    4    3    2    1    3   cvmx_sysinfo_get    1952 1990 1990 1990 1990 1990 1991 1985 1988 1984 1988 1988 1989 1990 1991 1989   main    1380 1990 1990 1990 1990 1990 1991 1985 1988 1984 1988 1988 1989 1990 1991 1989     application_main_loop    1178 1986 1987 1979 1978 1979 1982 1968 1983 1967 1968 1978 1980 1979 1981 1981       cvmx_pow_work_request_sync      15    3    1    4    3    2    0    5   11    0    4    1    2    5    3    0         cvmx_pow_tag_sw_wait    1163 1983 1986 1975 1975 1977 1982 1963 1972 1967 1964 1977 1978 1974 1978 1981         cvmx_pow_work_request_sync_nocheck      19    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0           cvmx_phys_to_ptr       3    0    1    3    2    1    0    0    0    2    1    0    2    0    0    0           cvmx_read_csr       3    0    1    3    2    1    0    0    0    2    1    0    2    0    0    0             cvmx_read64_uint64       7    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0     cvmx_fau_async_tagwait_fetch_and_add64       7    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0       cvmx_send_single       7    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0         cvmx_fau_iobdma_data       4    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0     cvmx_pko_get_base_queue     382    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0     cvmx_pko_send_packet_finish       6    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0       cvmx_phys_to_ptr      43    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0       cvmx_pko_doorbell       9    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0         cvmx_write_io       9    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0           cvmx_write64_uint64       8    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0       cvmx_scratch_read64       5    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0     cvmx_pow_tag_sw_nocheck       7    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0     cvmx_pow_tag_sw_wait      13    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0     cvmx_write_io      13    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0       cvmx_write64_uint64

For more detailed profiling over longer periods of time, the graphical result viewer perfzilla can be used. Instead of viewing instant profile results, profile information can be saved to a log file and viewed later. The output of oct-remote-profile can be redirected to a file.

To capture profiling data:

    $ sudo oct-remote-profile > profile.log

Then hit Control-C to stop capturing. The resulting data can be viewed with:

    $ perfzilla profile.log elf_file

Here is an example screen capture from profiling the Linux kernel.

14. Resetting Octeon (oct-remote-reset)

A soft reset of Octeon can be performed remotely using the oct-remote-reset utility. Here is an example:

    $ sudo oct-remote-reset    Found Octeon on bus 0 in slot 5. BAR0=0xf8000000, BAR1=0xf0000000    $

15. Saving Memory to a file (oct-remote-save)

A contiguous region of DDR may be saved to a file using the oct-remote-save utility. The example below saves the low 8 Mbytes of DDR to a file.

    $ sudo oct-remote-save 0x0 0x800000 low-mem.bin    Found Octeon on bus 0 in slot 5. BAR0=0xf8000000, BAR1=0xf0000000    $

16. Tracing using the Octeon Trace Buffer (oct-remote-tra)

L2 transactions and CSR accesses can be traced using oct-remote-tra.

$ ./oct-remote-tra -HCavium Octeon TRA AnalyzerUsage:    ./oct-remote-tra [options]Supported Options:    -h --help    -H --long-help    -f --filter=<match>        Set the filter for the events to record. The default        records nothing.    -w --wrap        Overwrite old samples when buffer overflows. When        samples are written faster than we can read them, the        new samples will overwrite old samples.    -m --trig-mode=[none|start|stop|both]        Set the mode of how triggers are used.        none = Triggers are ignored. This is the default.        start = Both triggers signal when TRA should start.        stop = Both triggers signal when TRA should stop.        both = Trigger 1 starts tracing, trigger 2 stops it.    -0 --trig0=<match>    -1 --trig1=<match>    -d --mcd0        Pulse MCD0 to stop cores when the TRA is full. This        gives this program a change to catch up. Once the TRA        buffer is emptied, the cores will be restarted.    -o --output=name        Write TRA output to binary file "name". This file can        be read later using the "-i" option.    -i --input=name        Read from binary file "name" instead of the hardware.        This should be used to read a file created with the "-o"        option.<match>=<commands>,<sources>,<destinations>,<address>,<mask><commands> is a list of the following separated by '+':    dwb, pl2, psl1, ldd, ldi, ldt, stc, stf, stp, stt, iobld8,    iobld16, iobld32, iobld64, iobst, iobdma, or all<sources> is a list of the following separated by '+':    dwb, iobreq, pko, pki, pp0-pp15, or all<destinations> is a list of the following separated by '+':    pow, rng, zip, dfa, fpa, key, pci, mio, or all<address> is a physical address to match<mask> Each bits set is a bit that is checked in <address>Brief description of the <commands>:    dwb     - Don't-Write-Back: Clear the dirty bit in the L2              tags if the cache block is present in the L2              cache. This is an ADD-only transaction on the CMB.    pl2     - Prefetch Into L2: The cache block will be put              into the L2 cache. This is an ADD-only              transaction on the CMB. These transactions are              only issued by cores.    psl1    - Data Cache Fill-Through: The cache block is filled              from L2/DRAM. The block will be filled into the              dcache of the requesting core. A subsequent store              from another core or IOB will cause an invalidate.              The block will not be put into the L2 cache. These              transactions can only be issued by cores.    ldd     - Data Cache Fill: The cache block is filled from              L2/DRAM. The block will be filled into the dcache              of the requesting core. A subsequent store from              another core or IOB will cause an invalidate. The              block is put into the L2 cache. These transactions              can only be issued by cores.    ldi     - Instruction Cache / IO Fill: The cache block is              filled from L2/DRAM. The block is put into the L2              cache. These transactions can be issued either by              cores or by the IOB. A subsequent store from              another core or IOB will NOT cause an invalidate.    ldt     - Instruction Cache / IO Fill-Through: The cache              block is filled from L2/DRAM. The block will not              be put into the L2 cache. These transactions can              be issued either by cores or by the IOB.    stc     - Store-Conditional: Store to either 32 or 64-bits              if the block is currently held in the data cache              of the requesting core. If the block is in the              data cache of the requesting core, the store              happens and a commit indication is returned. All              data cache copies of the block will be invalidated              in the other cores. If the block is not in the              data cache of the requesting core, a failure              indication is returned, no invalidate occurs,              neither does the store. The cache block will be              put into the L2 cache. These transactions are              only issued by cores.    stf     - Store-Full: Store to all bytes in the cache block.              The value of the bytes in the cache block that are              not transferred on the STORE bus or are masked off              will be written to zero. All data cache copies of              the block will be invalidated, except for the data              cache of an initiating core. The cache block will              be put into the L2 cache. These transactions are              issued either by cores or by the IOB.    stp     - Store-Partial: Some of the bytes in the cache              block will be stored. The value of the bytes in              the cache block that are not transferred on the              STORE bus or are masked off are not modified. All              data cache copies of the block will be invalidated,              except for the data cache of an initiating core.              The cache block will be put into the L2 cache.              These transactions are issued either by cores or              by the IOB.    stt     - Store-Through: Store to all bytes in the cache              block. The value of the bytes in the cache block              that are not transferred on the STORE bus or are              masked off will be written to zero. All data cache              copies of the block will be invalidated, except              for the data cache of an initiating core. The              cache block will not be put into the L2 cache.              These transactions are only issued by the IOB.    iobld8  - Load Reflection 8bit: An ADD cycle will be              reflected onto the FILL bus. This is used for              communication from the cores to the IOB. A load              reflection transaction is generated by the cores              to transfer a load to IOB.    iobld16 - Load Reflection 16bit.    iobld32 - Load Reflection 32bit.    iobld64 - Load Reflection 64bit.    iobst   - Store Reflection: Reflect an ADD cycle and              corresponding STORE cycle(s) onto the FILL bus.              This is used for core IOB communication. A store              reflection transaction is generated by the cores              to transfer a store/IOBDMA to IOB. An IOBDMA must              have exactly 64 bits of (aligned) STORE data. A              store may have 64, 32, 16, or 8 bits of aligned              STORE data. These transactions are destined to              IOB.    iobdma  - A store reflection transaction is generated              by the IOB to respond to a prior load/IOBDMA              request from a core. A load response always              contains exactly 64 bits of (aligned) STORE              data. An individual IOBDMA response contains              between one and sixteen 64 bit (aligned) words              of data. These transactions are destined to a core.    all     - All of the above commands added together.Brief description of the <sources>:    dwb     - Indicates a DWB from the DWB engine in IOB.    iobreq  - Indicates a FPA, TIM, DFA, PCI, ZIP, or POW              read/write, or a PKO write.    pko     - Indicates a PKO read.    pki     - Indicates a PIP/IPD write.    pp0-pp15- Indicates a particular core.    all     - All of the sources added together.Brief description of the <destinations>:    pow     - Indicates a destination of POW (get work, add              work, status/memory/index loads, NULLRd loads,              CSRs).    rng     - Indicates a destination of RNG (loads/IOBDMAs).    zip     - Indicates a destination of ZIP (doorbells).    dfa     - Indicates a destination of DFA (CSR's, doorbells,              and other operations).    fpa     - Indicates a destination of FPA (free pool              allocates / frees).    key     - Indicates a destination of KEY (reads/writes).    pci     - Indicates a destination of PCI (access to RSL-type              CSRs from many CN38XX blocks, PCI bus operations,              and PCI CSRs).    mio     - Indicates a destination of CIU or GPIO (for CSRs).    all     - Matches all destinations.Examples:    ./oct-remote-tra -f "ldi,all"        Trace all Icache loads.    ./oct-remote-tra -f "stc+stf+stp+stt,pp5+pp4,all"        Trace all stores from Core 4 and 5.    ./oct-remote-tra -f "stc+stf+stp+stt,all,all,0,0xffffffffffff0000"        Trace all stores to the first 64KB of memory (exception vectors).

17. Determining proper DDR configuration parameters (oct-remote-ddr-tune)

The DDR2 controller in Octeon processors provide a number of timing configuration parameters that vary from board to board. The oct-remote-ddr-tune program helps to determine proper settings of the board delay, LMC_DELAY[DQ], and LMC_DELAY[CLK]. The board delay parameter maps to several CSR fields (that vary between Octeon models), but is treated as a single configuration parameter. At a minimum, the oct-remote-ddr-tune program must be provided the SPD addresses for the DIMMS installed on the board. The range of parameters over which the testing is done can also be controlled.

$ oct-remote-ddr-tune --helpUsage: ./oct-remote-ddr-tune [options] [memtest filename]    Perform DDR tuning tests on remote Octeon.    --scantwsi: Scan twsi bus and exit.    --test_time=<seconds>: Duration to test each configuration. (Defaults to 10 seconds)    --ddr0spd=<twsi_addr[,twsi_addr]|filename[,filename]>: SPD addresses or files for DIMMS on interface 0.  Specify multiple times               if multiple dimms are present.  For 128 bit wide DDR interaces (CN38XX/CN58XX)               comma separated pairs off addresses must be used.    --ddr1spd=<twsi_addr|filename>: SPD addresses or files for DIMMS on DDR interface 1.  Only valid for Chips               with 2 DDR controllers.  Specify multiple times for multiple DIMMS.    --cpu_ref_hz=<NNN>: CPU reference clock in Hz.  Defaults to 50 MHz    --ddr_ref_hz=<NNN>: DDR reference clock in Hz.  Defaults to 50 MHz    --ddr_clock_hz=<NNN>: DDR clock in Hz. Defaults to 266 MHz. For 5XXX and newer chips, this controls the generated DDR clock.              For 3XXX chips, this must match the the configuration of the board.    --min_dq=<NNN>: minimum LMC_DELAY[DQ]. (defaults to 0, valid range 0-15)    --max_dq=<NNN>: maximum LMC_DELAY[DQ]. (defaults to 15, valid range 0-15)    --inc_dq=<NNN>: LMC_DELAY[DQ] incremment. (defaults to 1, valid range 1-15)    --min_clk=<NNN>: minimum LMC_DELAY[CLK]. (defaults to 0, valid range 0-15)    --max_clk=<NNN>: maximum LMC_DELAY[CLK]. (defaults to 15, valid range 0-15)    --inc_clk=<NNN>: LMC_DELAY[CLK] incremment. (defaults to 1, valid range 1-15)    --tuple_eeprom=<NNN>: EEPROM address for tuple eeprom to read board serial number from.    --readlevel:  Enable read leveling (auto board delay), CN52XX/CN56XX only.    --min_board_delay=<NNN>: minimum board delay, in picoseconds. (defaults to 0, valid range 0-3.5 DDR clock periods)    --max_board_delay=<NNN>: maximum board delay, in picoseconds. (defaults to approx 10000, valid range 0-3.5 DDR clock periods)For boards without SPD eeproms, filenames rather than SPD addresses may be specified.  The filesmust contain SPD values with 1 hex value per line, and a minimum of 64 values.Examples for some Cavium Networks boards: EBT3000: (two DIMMS on DDR interface 0, 128 bits wide)    oct-remote-ddr-tune --ddr0spd=0x50,0x51 --ddr_clock_hz=266000000 EBT5800: (two DIMMS on DDR interface 0, 128 bits wide)    oct-remote-ddr-tune --ddr0spd=0x50,0x51 --ddr_ref_hz=83250000 THUNDER:    oct-remote-ddr-tune --ddr0spd=0x51,0x52 --tuple_eeprom=0x53 NICPRO2:    oct-remote-ddr-tune --ddr0spd=0x51,0x52 --ddr0spd=0x55,0x56 --ddr_ref_hz=100000000 NIC_XLE_4G: (one DIMM one each DDR interface)    oct-remote-ddr-tune --ddr0spd=0x50 --ddr1spd=0x52 EBH5600: (two DIMMS on each interface)    oct-remote-ddr-tune --ddr0spd=0x50 --ddr0spd=0x51 --ddr1spd=0x52 --ddr1spd=0x53 Board with soldered on DRAM (no SPDs)    oct-remote-ddr-tune --ddr0spd=sdpfile --ddr1spd=spdfileNotes: CN3XXX chips do not have a programmable DDR clock PLL, and use a fixed multiplier. Only a multiplier of 4 is supported.  The correct DDR clock rate must be specified, but the reference clock rate is not used. The CN31XX does not support the DQ and CMD delays - only board delay can be adjusted. CN58XX and CN50XX chips have a DDR clock PLL, but can also be used with a variable reference clock. The correct reference clock must be specified for these chips.  The DDR clock is programmable via software. If the wrong reference clock is specified, the wrong DDR clock rate will be generated.  If the measured DDR clock rate is different than expected, an error message will be printed along with a suggested reference clock. CN56XX and CN52XX only support a 50 MHz DDR reference clock, and use a programmable PLL to generate the DDR clock.  The DDR clock is programmable via software.

oct-remote-ddr-tune creates 3 files when it runs: a results HTML document (ddr-tune-DATE.html), a log of the results in a parsable format (ddr-tune-DATE.log), and a debug log file (ddr-tune-debug-DATE.log.) The 'DATE' is a date string that is generated when the test is started. The debug log is rarely needed, and not discussed further.

Interpreting the results: The results HTML document contains a number of tables, one for each LMC_DELAY[CLK] that was used. Within each table, the results for each board delay/LMC_DELAY[DQ] are displayed. Red entries indicate that the memory test failed to run properly (failed to start, hung, etc.) Yellow entries indicate that errors were detected during the tests, and the number of errors per DDR test iteration are listed. Testing stops as soon as an error is detected, so the error numbers reported are normalized to be per-iteration. Green entries indicate that the test ran to completion without errors. These result tables will typically have a central green region, surrounded by yellow and red. Settings near the center of a green area area should be chosen. Note that the memory tests used to generate these tables are very short (seconds in length), so it is possible for the tables to have nice green sections at a given frequency, but have memory be unreliable at that frequency during long term testing.

For more details on tuning Octeon DDR2 interfaces, please refer to application note "AN-0067_OCTEON_Plus_DDR2_System_Tuning" that is available on the Cavium Networks support site. Please refer to the Octeon Hardware reference manual for more information on the memory controller register settings.

18. Debugging using GDB

For debugging an application over PCI, use the Simple Executive Debugger. This uses the oct-debug-agent to connect to the octeon board over PCI. This program is located in the host/debug-agent directory and depends on the host/remote-lib library like all other remote-utils applications. Once the application is built invoke the Simple Executive debugger as mentioned below:

    $ mipsisa64-octeon-elf-gdb -q passthrough    (Core#0-gdb) target octeonpci bootoct $(fileaddr) coremask=ffff    Sending bootloader command:  bootoct $(fileaddr) coremask=ffff debug    Listening on port: 44625    Connected to peer: 127.0.0.1    Remote target octeon connected to tcp::44625    (Core#0-gdb)

The boot command should be passed as an argument to the target octeonpci command. Note that debug option is automatically added to the boot command by GDB. This will make the program enter into the debug exception handler.

After the target command you can insert breakpoints and continue debugging.

    (Core#0-gdb) b main    Breakpoint 1 at 0x10000290: file passthrough.c, line 379.    (Core#0-gdb) set step-all on    (Core#0-gdb) set active-cores    (Core#0-gdb) c    Continuing.    Breakpoint 1, main () at passthrough.c:379    379         cvmx_user_app_init();    (Core#0-gdb)

Note:

Please refer to Simple Executive Debugger for the meaning of "set step-all on" and "set active-cores" commands.

These are the additional Cavium specific GDB commands that are supported only while debugging over PCI.

• show core-state: Displays the state of the core by dumping all the general purpose registers, COP0 registers and 32 TLB entries.

• set/show pci-bootcmd: A shell command for booting over PCI. The default value is set is "oct-remote-reset". The "target octeonpci" command basically resets the board before loading the program. For the boards that don't have boot flash, one needs to load the bootloader after reset. Pass the bootloader needed for PCI boot through "pci-bootcmd" before invoking "target octeonpci" command.

Here is an example on how to set pci-bootcmd.

    $ mipsisa64-octeon-elf-gdb -q passthrough    (Core#0-gdb) set pci-bootcmd oct-remote-boot    (Core#0-gdb) target octeonpci bootoct $(fileaddr) coremask=ffff    Found Octeon on bus 3 in slot 2. BAR0=0xe7fff000, BAR1=0xd8000000    Using bootloader image: /home/creese/work/octsw/sdk/target/bin/u-boot-pciboot-ebt3000.bin    Found Octeon on bus 3 in slot 2. BAR0=0xe7fff000, BAR1=0xd8000000    Sending bootloader command: bootoct $(fileaddr) coremask=ffff debug    0x10000018 in _init ()    (Core#0-gdb)

You can also attach to a program that is already running on the board by invoking the target octeonpci command with no arguments. After the target command the cores are stopped.

    $ mipsisa64-octeon-elf-gdb -q passthrough    (Core#0-gdb) target octeonpci    Found Octeon on bus 3 in slot 2. BAR0=0xe7fff000, BAR1=0xd8000000    (Core#0-gdb) bt    #0  main () at /home/anemet/octsw/sdk/target/include/cvmx-pow.h:360

The detach command or exiting GDB resumes execution of the program.

You can use a different remote protocol by changing how the target command is used. Example:

    $ mipsisa64-octeon-elf-gdb -q passthrough    (Core#0-gdb) target octeon LINUX

You can invoke oct-debug-agent manually to be able to debug oct-debug-agent or be able to run the debugger on another computer. Example:

    $ oct-debug-agent ./passthrough -q --target PCI --listen 38026 --pci-bootcmd oct-pci-boot -- bootoct 0 coremask=ffff

Usage: oct-debug-agent <exec_file> <option(s)> [<boot commands>] Listen on <port> for incoming connection fro GDB and perform the corresponding operation over the <comm> debug interface. Possible values for <option(s)> are:  -l, --listen <num>  Port number to listen for GDB connection  -t, --target <targetstr> (Required)   -d, --debug         Print debug messages during execution  -b, --pci-bootcmd <exec>  Set boot command needed for PCI booting  -q, --quiet         Quiet during execution  -h, --help          Display help message