Re: [PATCH 3/5] automation: Add the expect script with test case for FVP

[Henry Wang <Henry.Wang@xxxxxxx>]

Hi Stefano,

> On Dec 8, 2023, at 09:38, Stefano Stabellini <sstabellini@xxxxxxxxxx> wrote:
> 
> On Thu, 7 Dec 2023, Henry Wang wrote:
>> To interact with the FVP (for example entering the U-Boot shell
>> and transferring the files by TFTP), we need to connect the
>> corresponding port by the telnet first. Use an expect script to
>> simplify the automation of the whole "interacting with FVP" stuff.
>> 
>> The expect script will firstly detect the IP of the host, then
>> connect to the telnet port of the FVP, set the `serverip` and `ipaddr`
>> for the TFTP service in the U-Boot shell, and finally boot Xen from
>> U-Boot and wait for the expected results by Xen, Dom0 and DomU.
> 
> I am not an expert in "expect" but this script looks great

:)

> 
> 
>> Signed-off-by: Henry Wang <Henry.Wang@xxxxxxx>
>> ---
>> .../expect/fvp-base-smoke-dom0-arm64.exp      | 73 +++++++++++++++++++
>> 1 file changed, 73 insertions(+)
>> create mode 100755 automation/scripts/expect/fvp-base-smoke-dom0-arm64.exp
>> 
>> +
>> +proc test_boot {{maxline} {host_ip}} {
>> +    expect_after {
>> +        -re "(.*)\r" {
>> +            if {$maxline != 0} {
>> +                set maxline [expr {$maxline - 1}]
>> +                if {$maxline <= 0} {
>> +                    send_user "ERROR-Toomuch!\n"
>> +                    exit 2
>> +                }
>> +            }
>> +            exp_continue
>> +        }
>> +        timeout {send_user "ERROR-Timeout!\n"; exit 3}
>> +        eof {send_user "ERROR-EOF!\n"; exit 4}
>> +    }
> 
> Why do we need this "expect_after" ?

It is basically for the error handling. Either there is too many characters 
triggered
by some misconfiguration of Xen/Kernel leading to Xen/Kernel constantly reboots,
or the code stuck somehow, we have a way to stop the script instead of waiting 
in
the infinity loop.

>> +    # Extract the telnet port numbers from FVP output, because the telnet 
>> ports
>> +    # are not guaranteed to be fixed numbers.
>> +    expect -re {terminal_0: Listening for serial connection on port [0-9]+}
>> +    set terminal_0 $expect_out(0,string)
>> +    if {[regexp {port (\d+)} $terminal_0 match port_0]} {
>> +        puts "terminal_0 port is $port_0"
>> +    } else {
>> +        puts "terminal_0 port not found"
>> +        exit 5
>> +    }
>> +
>> +    spawn bash -c "telnet localhost $port_0"
>> +    expect -re "Hit any key to stop autoboot.*"
>> +    send -s "  \r"
>> +    send -s "setenv serverip $host_ip; setenv ipaddr $host_ip; tftpb 
>> 0x80200000 boot.scr; source 0x80200000\r"
>> +
>> +    # Initial Xen boot
>> +    expect -re "\(XEN\).*Freed .* init memory."
>> +
>> +    # Dom0 and DomU
>> +    expect -re "Domain-0.*"
>> +    expect -re "BusyBox.*"
>> +    expect -re "/ #.*"
> 
> This is clear, excellent
> 
>> +}
>> +
>> +# Get host IP
>> +spawn bash -c "hostname -I | awk '{print \$1}'"
>> +expect -re {(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})}
> 
> Why d{1,3}?

I think that is because the IP pattern, we at least have one digit and at most 
3.

>> +set host_ip $expect_out(0,string)
>> +
>> +# Start the FVP and run the test
>> +spawn bash -c "$runcmd"
>> +
>> +test_boot 2000 "$host_ip"
>> +
>> +send_user "\nExecution with SUCCESS\n"
> 
> Won't this always return SUCCESS even in case of failure?

IMHO, if things fails, we have various exit code (1-5) for each failure case. 
For example,
if the FVP port somehow cannot be found, we exit with error code 5. This will 
basically lead
us to the only case where the failure is caused by the script fails to wait for 
the expected
string/regexp, and this case we have the timeout failure triggered by my 
above-mentioned
expect_after block.

Kind regards,
Henry

>> +exit 0
>> -- 
>> 2.25.1
>>