[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH v2 3/5] automation: Add the expect script with test case for FVP
- To: Michal Orzel <michal.orzel@xxxxxxx>
- From: Henry Wang <Henry.Wang@xxxxxxx>
- Date: Fri, 8 Dec 2023 09:05:07 +0000
- Accept-language: zh-CN, en-US
- Arc-authentication-results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 63.35.35.123) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com] dmarc=[1,1,header.from=arm.com])
- Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none
- Arc-message-signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=uG8wieN9a2R+nD+GSP5TqEbozMuXdUaOHljYikQtWZM=; b=RmWYN/Kgu2Nz1L/pwfQ4zF0PNkypNgjPpPFADiuCyCNp6Pi5/ViyzXmr4gnIGkUCqwJ1eKsRMUPKx8J6yH3BtHTGeF0wo5K1koJwu6sfQ9lETozq2wJQbC7/FBcnHF4EeVFo81oEktyIgHig5ahOpxn0sS7+WLfF4gbVU3hJ5mkDDyavCVy3F2rxvy80t46f1vsUgxGtnOq/GpzYQqdqorvdiuNujfFDFrNDYBYM2HuC/DMOH2sAMEwCSjmckZS5EIIXcSuxJohAzboXbyglmXPqWvVod8tSRLBrQhKbkKCkzGuwxOHzb3c2kLCUK/XFyKXT0VFDcp1rr1FyALvVxw==
- Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=uG8wieN9a2R+nD+GSP5TqEbozMuXdUaOHljYikQtWZM=; b=mU9J/RaYOjOIjXZvpE22wdXyn6YFQkTsmsAASkWXlJ0/KlZAKSqsI95uBUR3x4P/wysEVLLqLZYUw9bCIQdPSWCYyh5TIolNFPDlQ/nWzcUdEJpqrurL7p87cLa73Xi/ZEVOZZL8MFBQ8yKP6cL+VMBcGIQ03lGmvM9CkET4NUQkv4mpja8NGkIP6taseqOm45NNrgEAWcd8YhUkSUMHP2RLjm5aFBV5mq1Xz4ag2tA/sZA1fkb1/XjFZtIyMv0tGzraCtlq9BxpYSoG8jNUsDH2Zs9hG7qS4NPrqtlk1zaxkrQ0lYc+iy2UYI5Kol8/bmL2c0EsI438QD8gD86/xg==
- Arc-seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass; b=MaVXAC9xCKVQS6fwxnbd1BxrtN+5FsEWG0CthMsNGczLoj0SpxgM9zx8IwGRAHuQ07MzPPbooGS6xmqsLiHgKvPgH2qy2UL0quW0eqs6/b5NEdWxWGaEfFbzCkRX0p/+7iyUBw9/AB/SNDFoGqdU0W6U2oiIzeGJQRJdG5UiMKzcqh6b2mPsKJ7MmdfMORqmDi2XjXc8mKQTmsAVnqlGGu6GBiTUrK06Mz8EgxBHTrnbz5LcLJc76qz4te9LuJc/r0LL0jY3TVUrWoRvsISx/R59HC636NLVz8aN7azN1FiJoRH2KY9nRp9wi0V2w/vI2Ohlv+OGOgq0856k5NvsDQ==
- Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FK189mmTq3bgXRhNTyvY/4mOu3o0twPQjrg1iXcfl9nSG2j5b9DWLAjMv1TNLJI34iTTJNMGflZxDBpJGfZ3j1jiwnifhIeoIAWcxFK9U5dSsFcL5BJR17BPmDMqLbLlfAKX/QgWQiFkg2W9EAVu3l7ixRQMEmP6GnSZ8peGOWatxcJL6cdC5rFL6qChzdVZB/Zw3SQzEWsxjddSGAxNIBWjpWICan6WWOiMG7LTzV438fmyv4T+734h9nS91uuzHacKMB8j1dVTN7eoCX9Qqxm7EhVDeKDnV4/+XtA5LbnKJAhECWNzZsdBNMb+IsGa0ohmGb9gNUjlZcsC4ltxJA==
- Authentication-results-original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com;
- Cc: Xen-devel <xen-devel@xxxxxxxxxxxxxxxxxxxx>, Doug Goldstein <cardoe@xxxxxxxxxx>, Stefano Stabellini <sstabellini@xxxxxxxxxx>, Julien Grall <julien@xxxxxxx>, Bertrand Marquis <Bertrand.Marquis@xxxxxxx>, Wei Chen <Wei.Chen@xxxxxxx>
- Delivery-date: Fri, 08 Dec 2023 09:05:23 +0000
- List-id: Xen developer discussion <xen-devel.lists.xenproject.org>
- Nodisclaimer: true
- Original-authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com;
- Thread-index: AQHaKZoAVIHO2rCYDECWUL8sUR2sfLCfFe6AgAACFwA=
- Thread-topic: [PATCH v2 3/5] automation: Add the expect script with test case for FVP
Hi Michal,
> On Dec 8, 2023, at 16:57, Michal Orzel <michal.orzel@xxxxxxx> wrote:
>
> Hi Henry,
>
> On 08/12/2023 06:46, Henry Wang wrote:
>>
>>
>> To interact with the FVP (for example entering the U-Boot shell
>> and transferring the files by TFTP), we need to connect the
>> corresponding port by the telnet first. Use an expect script to
>> simplify the automation of the whole "interacting with FVP" stuff.
>>
>> The expect script will firstly detect the IP of the host, then
>> connect to the telnet port of the FVP, set the `serverip` and `ipaddr`
>> for the TFTP service in the U-Boot shell, and finally boot Xen from
>> U-Boot and wait for the expected results by Xen, Dom0 and DomU.
>>
>> Signed-off-by: Henry Wang <Henry.Wang@xxxxxxx>
> Reviewed-by: Michal Orzel <michal.orzel@xxxxxxx>
Thanks!
> with 1 question...
>
>> ---
>> v2:
>> - No change.
>> ---
>> .../expect/fvp-base-smoke-dom0-arm64.exp | 73 +++++++++++++++++++
>> 1 file changed, 73 insertions(+)
>> create mode 100755 automation/scripts/expect/fvp-base-smoke-dom0-arm64.exp
>>
>> diff --git a/automation/scripts/expect/fvp-base-smoke-dom0-arm64.exp
>> b/automation/scripts/expect/fvp-base-smoke-dom0-arm64.exp
>> new file mode 100755
>> index 0000000000..25d9a5f81c
>> --- /dev/null
>> +++ b/automation/scripts/expect/fvp-base-smoke-dom0-arm64.exp
>> @@ -0,0 +1,73 @@
>> +#!/usr/bin/expect
>> +
>> +set timeout 2000
> Do we really need such a big timeout (~30 min)?
> Looking at your test job, it took 16 mins (quite a lot but I know FVP is slow
> + send_slow slows things down)
This is a really good question. I did have the same question while working on
the negative test today. The timeout 2000 indeed will fail the job at about
30min,
and waiting for it is indeed not really pleasant.
But my second thought would be - from my observation, the overall time now
would vary between 15min ~ 20min, and having a 10min margin is not that crazy
given that we probably will do more testing from the job in the future, and if
the
GitLab Arm worker is high loaded, FVP will probably become slower. And normally
we don’t even trigger the timeout as the job will normally pass. So I decided
to keep this.
Mind sharing your thoughts about the better value of the timeout? Probably
25min?
Kind regards,
Henry
>
> ~Michal
>
|
