Adding Natural‑Language UI Tests with Amazon Nova Act: What Epsilon Did and What It Means for You

Epsilon, the data‑and‑marketing arm of Publicis Groupe, needed a way to give dozens of product teams a UI‑testing service that wouldn’t force each team to set up its own browser farm. By wiring Amazon Nova Act to the Bedrock AgentCore Browser tool, they delivered a multi‑tenant, natural‑language test runner that sits alongside their existing Playwright suite.

---

The brittleness of selector‑based regression suites

Epsilon’s engineers already ran a Playwright regression suite that checked functionality across many web apps. Playwright works well for deterministic checks, but its tests are tightly bound to CSS selectors and page structure. A harmless cosmetic change – a class rename, a layout tweak, or a new banner – can cause a test to fail even though the core feature still works. Fixing those failures is a manual, repetitive task that scales poorly when dozens of teams each maintain their own test harness.

---

How Nova Act and the Bedrock Browser tool make natural‑language steps possible

Nova Act is a managed service that accepts plain‑English instructions, translates them into actions over a Chrome DevTools Protocol (CDP) WebSocket, and returns a structured result. The Bedrock AgentCore Browser tool supplies an on‑demand, head‑less Chromium instance, handles session recording, and integrates with AWS IAM for authentication. Together they let a test look like:

Confirm the rewards balance is visible

instead of a line of Playwright code that locates a specific DOM node. The language model behind Nova Act decides which element matches the intent, clicks it, or scrolls until it appears, handling conditional logic such as “if a tour box appears, skip it.” Because the browser runs in an isolated AWS‑managed container, teams do not have to ship binaries or maintain a display server.

---

Epsilon’s platform service: three layers, one API

Epsilon wrapped the Nova Act + Browser combo in a multi‑tenant service that looks like a normal agent endpoint. The flow is:

1. API call – a product team posts a workflow definition (target URL + list of intent steps) to the platform API.
2. Workflow registration – the platform registers the definition with Nova Act.
3. Background runner – a Celery worker pulls the job from SQS, opens a Bedrock Browser session, and streams each step to Nova Act over CDP.
4. Result capture – screenshots, pass/fail flags, and a JSON payload are stored in S3; a webhook posts the summary back to the caller.

The architecture diagram in the source shows an API Gateway → Django REST → ECS runner → Nova Act + Browser → S3/CloudTrail. Two skills expose the capability:

• ui‑test‑recorder – interactive recording of intent steps.
• ui‑test‑executor – execution of a stored trajectory.

Because each tenant runs under its own AWS STS‑AssumeRole, isolation and auditability are baked in without extra plumbing.

---

The hidden trade‑off: LLM cost, latency, and test stability

The obvious win is reduced flakiness from cosmetic changes. The downside is that every step incurs a call to the underlying LLM, which adds latency and a variable AWS cost (token‑based pricing for Bedrock models). In practice Epsilon mitigates the cost by replaying recorded trajectories for regression runs, skipping the LLM “thinking” phase once a step has been successfully resolved. That optimization works well for stable flows but adds complexity: you must version‑track trajectories and decide when to invalidate them after a UI redesign.

Another subtle point is viewport handling. The Bedrock Browser starts with a default viewport size; steps that rely on scrolling must be phrased to account for that. If a test says “scroll down until the Goal Statement field is visible,” Nova Act will succeed only if the field eventually appears within the viewport. A change that pushes the field off‑screen beyond the default size can cause a false failure.

Overall, the trade‑off is between human‑readable, low‑maintenance test scripts and higher per‑execution cost and latency. Teams with frequent UI churn and limited QA resources tend to benefit; teams that run thousands of cheap regression runs per day may find the added expense prohibitive.

---

Quick start: add an intent‑driven test to your own project today

1. Create an AWS account (or use an existing one) and enable Amazon Bedrock.
2. Install the Nova Act Python client:

   pip install nova-act
   

3. Spin up a Browser session in a short script:

   import boto3
   from bedrock_agentcore.tools.browser_client import browser_session
   from nova_act import NovaAct
   
   boto = boto3.Session(region_name="us-east-1")
   with browser_session("us-east-1") as client:
       ws_url, hdrs = client.generate_ws_headers()
       with NovaAct(cdp_endpoint_url=ws_url, cdp_headers=hdrs, boto_session=boto, starting_page="https://example.com") as act:
           result = act.act("Confirm the login button is visible")
           print(result)
   

4. Record a trajectory using the ui-test-recorder skill (the source shows an interactive UI, but a simple loop over a list of English steps works for a proof‑of‑concept).
5. Run it repeatedly – on subsequent runs, store the JSON result and replay it by feeding the same steps back to Nova Act, skipping LLM inference if the step succeeded before.

By following these five steps you’ll have a tiny, intent‑driven test that survives a button‑color change or a minor layout tweak without touching any selector code.

---

Sources

• Building intent‑driven UI testing with Amazon Nova Act (AWS Blog)