Newsletter curation agent

This page explains how a newsletter curation agent sweeps public industry sources under a silent grant, deduplicates against the published list in your content database, and creates a curation draft awaiting editor confirmation; the sweep is a long-running task you can reattach to at any time with run.id. After reading it, you will understand when this scenario actually needs interactive consent, how candidates are collected and deduplicated, and why data like published URLs belongs in your content database rather than Memory.

Use case

Marketing teams need to regularly curate newsletters from product updates, blogs, events, industry news, and customer stories. Candidate sources are scattered and update frequently, so manual source-by-source checks are slow and incomplete — and the same piece resurfaces through different syndication channels, requiring source-level deduplication.

Typical triggers:

• A newsletter issue deadline approaches, and a candidate list and section drafts are needed.
• A noteworthy industry development appears, and its fit for this issue must be judged quickly.
• Section structure or selection criteria change, and candidates must be re-filtered against the new configuration.

Engineering challenges

• The sweep is a long task spanning hours or days: sources are many and update on their own schedules, so one collection run can straddle deployment restarts and cron windows. The task handle must be persistable and reattachable — a process exit cannot mean starting over.
• Syndication dedupe needs a deterministic basis: the same piece carries different URLs and titles across channels, so deduplication must key on normalized URLs while keeping the original source — otherwise the same content reappears issue after issue.
• Unsourced content slips into the pool: aggregator and syndication pages often lose the original attribution, and candidates without an accessible link and collection time cannot support editorial decisions — they must be dropped before entering the pool.

Module composition

Module	Role	Notes
GenAuth	Core	Silent delegation issues the short-lived runtime credential every product call requires; interactive consent is only needed when login state or write actions enter the picture. Credentials remain short-lived, revocable, and audited.
Web Agent	Core	Sweeps public blogs, news, and industry sources with WebSearch, keeping a link and collection time per candidate; long runs persist run.id for reattachment.
GUMem	Not used	The published-URL list and section layout are business state — keep them in your content database, managed per issue. They are not Memory. This scenario has no personal user preference worth persisting across tasks.

When interactive consent is needed

Every product call requires a GenAuth delegate token; public read-only scenarios are covered by silent delegation. The default path here only reads public web pages: your app exchanges the GenAuth user ID bound to your Qoni credentials for a runtime credential, with no user redirect. The credential is explicit, short-lived, and revocable, covering only public reads and task execution; sending, bulk mailing, and subscriber operations sit outside every grant.

Upgrade to interactive consent (mode: 'interactive') when:

• Candidate collection needs a signed-in content pool, CMS, or paid subscription source.
• The task performs write actions, such as saving the curation draft into a CMS draft box.

The upgrade works the same as in other scenarios: mode: 'interactive' plus a redirectUri, with the user confirming in Qoni Console and your server exchanging the grant via completeDelegateToken — see the Quickstart for the full flow.

Workflow

1. The user selects a newsletter issue, topics, and candidate source range.

2. Your app obtains a runtime credential through a silent grant (public reads, no user redirect).

3. Web Agent starts the long-running public source sweep; your app persists run.id and reattaches after process restarts or at the next cron window.

4. The sweep returns a candidate list, each entry with an accessible link and collection time.

   Checkpoint: Every candidate should carry an accessible source link and collection time; unsourced entries do not enter the candidate pool.

5. Your app loads the published-URL list and section layout from the content database and injects them into the curation task.

6. The Agent deduplicates candidates by normalized URL, merges syndicated copies while keeping the original source, and drafts section ideas, summaries, and headlines against the section layout.

7. The Agent returns the curation draft and source list with an audit id; after the editor confirms, include and exclude decisions are written back to your content database.

   Checkpoint: Drafts stop at "awaiting confirmation"; any action toward sending or bulk mailing should be rejected and recorded.

Example code

This example uses @qoniai/qoni 0.9.0, published on npm. The download includes the same SDK version, installed with npm ci. The demo curates the Firefox page and Mozilla Manifesto for browser and open-web sections. The SDK reads real public pages; business inputs in scenarios.ts are labeled public-demo.

This demo does not read or write GUMem; its task uses the supplied page list and explicit business inputs. DoAnything opens the supplied pages and produces the scenario output. Supply your own JSON with --input; use --interactive when the user must consent to delegation. For site sign-in and user responses, see Qoni SDK.

Download the complete runnable examples, or run from the documentation repository:

cd examples/qoni
npm ci
npm run case -- newsletter-curation-agent
# Supply your own inputs
npm run case -- newsletter-curation-agent --input /path/to/input.json

Set server-side QONI_ACCESS_KEY and QONI_SECRET_KEY. QONI_USER_ID can identify your application's current GenAuth user; the local demo otherwise selects a user from the bound pool. Demonstration Memory writes use an isolated user rather than changing a business user's preferences.

This scenario's executable entry point:

import { cliOptions } from '../runtime.js'
import { inputFile, runScenario } from '../run-case.js'

// Select sourced public material for newsletter editorial drafts.
const report = await runScenario('newsletter-curation-agent', cliOptions(), inputFile())
// report.items: sections, titles, summaries, sources, and collection timestamps.
// Validated fields: section, title, summary, sourceUrl, collectedAt.
console.log(JSON.stringify(report, null, 2))

The entry point loads the definition below by scenario ID. The code is included directly from scenarios.ts, with comments shown in the page language: the task, output fields, source field, Web Search queries (if any), whether Memory is used, and the demonstration input. The pipeline appends the input data, search sources, recalled Memory, and shared safety constraints to the task to build the final prompt; see the pipeline below for the full assembly.

// Select sourced public material for newsletter editorial drafts.
browser('newsletter-curation-agent',
  'Read the supplied official public pages and curate two newsletter entries. Return [{section,title,summary,sourceUrl,collectedAt}]. Each entry needs a real source. Never send a newsletter, alter a subscriber list, or duplicate URLs marked alreadyPublished.',
  ['section','title','summary','sourceUrl','collectedAt'], 'sourceUrl', sample([firefox,manifesto], { sections:['browser','open-web'], alreadyPublished:[] })),

The sample implements runScenario(), browser(), research(), and sample() as application functions. The pipeline below makes the actual SDK calls: delegation and introspection → required Memory/search → browser task or monitor → validation and saving. fields and sourceField define the application's output checks. The complete application helpers are in the package's runtime.ts.

Inspect the actual SDK pipeline

// browser() uses DoAnything; research() searches first; sample() labels public-demo inputs.
const firefox = 'https://www.mozilla.org/en-US/firefox/new/'
const manifesto = 'https://www.mozilla.org/en-US/about/manifesto/'
const privacy = 'https://www.mozilla.org/en-US/privacy/firefox/'
const support = 'https://support.mozilla.org/en-US/kb/get-started-firefox-overview-main-features'
// These copy rules become prompt context; server permissions and business checks remain separate.
const policy = {
  version: 'demo-2026-09',
  approvedClaims: ['Describe only features supported by the cited page.'],
  forbiddenClaims: ['guaranteed security', '100% private', 'unverified pricing or performance'],
  voice: 'concise and warm',
}
const sample = (pages: string[], business: JsonObject = {}): JsonObject => ({
  dataset: 'public-demo', pages, policy, business,
  notice: 'Business records are synthetic demonstration inputs. Referenced websites and SDK execution are real.',
})
// fields lists required output keys; sourceField identifies URL checks; memory enables GUMem calls.
const browser = (id: string, task: string, fields: string[], sourceField: string | undefined, input: JsonObject, memory = false): Scenario => ({
  id, products: ['doAnything'], task, fields, sourceField, input, memory,
})
const research = (id: string, task: string, fields: string[], sourceField: string, queries: string[], input: JsonObject, memory = false): Scenario => ({
  id, products: ['webSearch', 'doAnything'], task, fields, sourceField, queries, input, memory,
})

import { QoniScopes, type JsonObject, type RunResult } from '@qoniai/qoni'
import { readFileSync } from 'node:fs'
import { getScenario, type Scenario } from './scenarios.js'
import { appendTrace, checkInputCoverage, cleanupDemoUser, createContext, delegate, handleInteraction, inputEntryCount, isolateDemoUser, object, readWithRetry, renderScreenshot,
  save, saveArtifacts, searchHits, settled, settleRun, validateItems, withCleanup, type Context, type Options } from './runtime.js'

export async function runScenario(id: string, options: Options = {}, input?: JsonObject) {
  // Load the task definition by ID; --input replaces its business inputs.
  const scenario = getScenario(id)
  const data = input ?? scenario.input
  if (scenario.memory && data.dataset !== 'public-demo' && options.mode !== 'interactive' && !options.userId && !process.env.QONI_USER_ID) {
    throw new Error('Business Memory writes require the current QONI_USER_ID; do not select an arbitrary bound user')
  }
  const context = await createContext(id, { ...options,
    skipUserResolution: scenario.memory && data.dataset === 'public-demo' })
  return withCleanup(context, async register => {
    register('isolated demonstration user', () => cleanupDemoUser(context))
    // Isolate demo preferences; business Memory belongs to the identified current user.
    if (scenario.memory && data.dataset === 'public-demo') await isolateDemoUser(context)
    return await executeScenario(context, scenario, data)
  })
}

export async function executeScenario(context: Context, scenario: Scenario, input: JsonObject) {
  const pages = input.pages
  if (!Array.isArray(pages) || !pages.length || pages.some(page => typeof page !== 'string' || !/^https:\/\//.test(page))) {
    throw new Error('Input pages must be an array of HTTPS URLs')
  }
  if (input.requiresLogin === true && context.mode !== 'interactive') {
    throw new Error('Targets that require sign-in need --interactive and user-controlled login')
  }
  const memoryScopes = scenario.memory
    ? [QoniScopes.GUMEM_MEMORY_READ, QoniScopes.GUMEM_MEMORY_WRITE, QoniScopes.GUMEM_MESSAGE_WRITE] : []
  // delegate() is an application helper around the SDK delegation methods.
  const grant = await delegate(context, scenario.id, scenario.products, memoryScopes)
  // Read the effective scopes; readWithRetry() retries only retryable read failures.
  const { data: tokenInfo } = await readWithRetry(context, 'delegation introspection',
    () => context.qoni.genauth.introspectDelegationToken({ token: grant.token }))
  const info = object(tokenInfo)
  if (info.active !== true) throw new Error('The delegation token is not active')
  const audit = { grantId: grant.grantId, auditId: grant.auditId, scopes: info.scope }
  let memory: unknown
  if (scenario.memory) {
    // A Session associates this conversation with the user; the app chooses sessionId.
    const sessionId = `${scenario.id}-${Date.now()}`
    await context.qoni.gumem.createSession({
      token: grant.token, userId: context.userId, sessionId, title: scenario.id,
    })
    const preferences = object(input.business ?? {}).confirmedPreferences
    if (Array.isArray(preferences) && preferences.length) {
      // Store confirmed preferences only; sync: true requests synchronous processing.
      await context.qoni.gumem.addMessages({ token: grant.token, userId: context.userId, sessionId, sync: true,
        messages: [{ role: 'user', content: `Confirmed demonstration preferences: ${preferences.join('; ')}` }] })
    }
    // Recall relevant preferences for the later task prompt.
    memory = (await readWithRetry(context, 'GUMem recall', () => context.qoni.gumem.recall({ token: grant.token, sessionId,
      query: 'Confirmed preferences relevant to this task', details: true }))).data
    save(context, 'memory.json', { sessionId, context: memory })
    if (Array.isArray(preferences) && preferences.length && !preferences.every(value => JSON.stringify(memory).includes(String(value)))) {
      throw new Error('Recall did not include the confirmed preferences just written by this demo')
    }
  }

  if (scenario.products.includes('track')) return runMonitor(context, scenario, input, grant.token, audit)

  let hits: ReturnType<typeof searchHits> = []
  if (scenario.queries) {
    // Web Search returns results[]; DoAnything receives these sources to inspect.
    const search = await context.qoni.webSearch.run({ token: grant.token, prompt: scenario.queries, maxResultsPerQuery: 3 })
    save(context, 'search-ref.json', { runId: search.id, audit })
    await withCleanup(context, async register => {
      register('Web Search run', () => search.cancel('Documentation demonstration cleanup'))
      const result = await settleRun(context, search)
      save(context, 'search-result.json', result)
      settled(result)
      hits = searchHits(result.output)
    })
  }

  // One-per-entry scenarios request exactly one item per input entry; others at most two.
  const requiredItems = inputEntryCount(scenario.id, input)
  // Assemble the task, inputs, search sources, and Memory as application-defined context.
  const prompt = [scenario.task, `Task inputs: ${JSON.stringify(input)}`,
    `Search sources: ${JSON.stringify(hits)}`, `Confirmed memory: ${JSON.stringify(memory ?? null)}`,
    `Actual collection time: ${new Date().toISOString()}`,
    requiredItems === undefined
      ? 'Inspect the supplied sources. Return at most two items in the requested JSON array, without prose or Markdown.'
      : `Inspect the supplied sources. Return exactly ${requiredItems} item${requiredItems === 1 ? '' : 's'} in the requested JSON array, one per input entry, without prose or Markdown.`,
    'Keep synthetic demonstration data identified as synthetic. Do not send messages, publish, pay, edit accounts or submit forms.',
    input.requiresLogin === true ? 'Request user sign-in through an interaction when required; never enter credentials yourself.' : 'Public demonstration sources only; do not sign in.',
  ].join('\n\n')
  // Start the Agent with this grant; capture receives delivered screenshots, not every step.
  const run = await context.qoni.doAnything.run({ token: grant.token, prompt, capture: { screenshots: true } })
  save(context, 'run-ref.json', { runId: run.id, session: run.sessionRef, audit })
  let result: RunResult
  const trace = (event: { type: string; data: unknown }) => {
    context.eventCounts[event.type] = (context.eventCounts[event.type] ?? 0) + 1
    if (['progress','message','done'].includes(event.type)) appendTrace(context, event)
    if (event.type === 'browserLiveUrlChanged') {
      const liveUrl = object(event.data).liveUrl
      if (typeof liveUrl === 'string') context.browserUrl = liveUrl
    }
  }
  return withCleanup(context, async register => {
    register('DoAnything run', () => run.cancel('Documentation demonstration cleanup'))
    if (context.delivery === 'events') {
      // --events streams updates; wrap interaction data in an SDK handle for user handling.
      for await (const event of run.events({ signal: AbortSignal.any([context.abort.signal, AbortSignal.timeout(context.timeoutMs)]) })) {
        trace(event)
        if (event.type === 'screenshot') renderScreenshot(context, event.image)
        if (event.type === 'interaction') await handleInteraction(context, run.interactionHandle(event.data))
      }
      result = await settleRun(context, run)
    } else {
      // Callback mode receives this run's events inside wait; helpers save images and ask the user.
      result = await settleRun(context, run, { onEvent: trace,
        onScreenshot: (image, index) => renderScreenshot(context, image, index),
        onInteraction: interaction => handleInteraction(context, interaction) })
    }
    save(context, 'result.json', result)
    settled(result)
    await saveArtifacts(context, result)
    if (scenario.id === 'landing-page-audit-agent' && context.screenshots === 0) throw new Error('The landing-page audit did not deliver the requested screenshot')
    // The app checks required fields and source URL formats; a reviewer still checks facts.
    const items = validateItems(result.output, scenario.fields, scenario.sourceField)
    // Scenarios that require one item per input entry are checked against the input.
    checkInputCoverage(scenario.id, items, input)
    const report = { scenario: scenario.id, dataset: input.dataset, passed: true, runId: run.id,
      status: result.status, items, audit, screenshots: context.screenshots, interactions: context.interactions,
      events: context.eventCounts, artifactIds: result.artifacts.map(artifact => artifact.id) }
    save(context, 'report.json', report)
    return report
  })
}

async function runMonitor(context: Context, scenario: Scenario, input: JsonObject, token: string, audit: JsonObject) {
  // Track creates a monitor with targets, extraction fields, and hourly scheduling.
  const monitor = await context.qoni.track.create({ token, prompt: scenario.task,
    targetUrls: input.pages, extractionSchema: { heading: 'string', source_url: 'string' },
    tickInstructions: `Open the target URLs and read the actual visible heading. Return a JSON object with heading and source_url. ${scenario.task}`,
    triggerDsl: { on: 'change' }, schedule: { kind: 'interval', intervalSeconds: 3600 } })
  save(context, 'monitor-ref.json', { id: monitor.id, audit })
  return withCleanup(context, async register => {
    register('Track monitor', () => monitor.delete())
    const definition = await monitor.get()
    save(context, 'monitor-definition.json', definition)
    if (object(definition.schedule).intervalSeconds !== 3600) throw new Error('Track did not persist the requested schedule interval')
    // Run one tick and inspect its extraction by runId; completed alone does not prove success.
    const tick = await monitor.runNow()
    save(context, 'tick.json', tick)
    if (tick.state !== 'completed') throw new Error(`Track execution failed: ${tick.state} / ${tick.error ?? ''}`)
    const runId = tick.runId
    if (typeof runId !== 'string') throw new Error('Track tick did not return a runId')
    const detail = await monitor.run(runId)
    save(context, 'tick-detail.json', detail)
    if (detail.state !== 'completed' || !detail.extracted || !Object.keys(object(detail.extracted)).length) {
      throw new Error('Track did not extract page data')
    }
    const extracted = object(detail.extracted)
    if (typeof extracted.heading !== 'string' || !extracted.heading.trim() ||
      typeof extracted.source_url !== 'string' || !/^https:\/\//.test(extracted.source_url)) {
      throw new Error('Track extraction is missing a heading or source URL')
    }
    const normalizeUrl = (value: string) => { const url = new URL(value); url.hash = ''; return url.href.replace(/\/$/, '') }
    if (!(input.pages as string[]).some(url => normalizeUrl(url) === normalizeUrl(String(extracted.source_url)))) {
      throw new Error('The Track source URL is not a configured target')
    }
    // Check persisted pause/resume state; withCleanup() deletes the monitor on exit.
    await monitor.pause()
    if ((await monitor.get()).status !== 'paused') throw new Error('Track did not persist the paused state')
    await monitor.resume()
    if ((await monitor.get()).status !== 'active') throw new Error('Track did not persist the active state')
    const report = { scenario: scenario.id, dataset: input.dataset, passed: true, monitorId: monitor.id,
      runId, state: detail.state, outcome: detail.outcome, extracted: detail.extracted, audit }
    save(context, 'report.json', report)
    return report
  })
}

export function inputFile(): JsonObject | undefined {
  const index = process.argv.indexOf('--input')
  return index >= 0 ? object(JSON.parse(readFileSync(process.argv[index + 1], 'utf8'))) : undefined
}

Results are written to output/newsletter-curation-agent/report.json. report.items contains sections, titles, summaries, sources, and collection timestamps, with fields section, title, summary, sourceUrl, collectedAt. audit links the grant ID, audit ID, and effective scopes; http.json records redacted request statuses. The application parses DoAnything output and checks required fields and source URL formats. A business reviewer still assesses the content against the original sources.

Data and memory boundaries

This scenario touches four kinds of data; GUMem is not used here:

• Versioned configuration: section layout and selection criteria — managed per issue in your content database or config store, with drafts referencing the configuration version.
• Business state: candidate lists, published URLs, and the editor's include and exclude decisions — written back to your content database for next-issue dedupe and retrospectives.
• Audit records: the delegation and behavior chain formed by grantId and auditId — maintained by GenAuth.
• User Memory (optional): only long-term personal preferences a user has explicitly confirmed belong in GUMem; selection criteria and section layout are team-level configuration, not personal memory, so this scenario neither recalls nor writes back by default.

Failure handling

Situation	Recommended handling
Process restart or wait() timeout	Reattach with the persisted run.id; the task keeps running server-side, so never start a duplicate.
The credential has expired at reattach time	Obtain a fresh silent grant, then attach to the same run.id.
A candidate source is unreachable or the content is taken down	Remove it from the candidate pool with the reason recorded; never cite content that cannot be verified.
An entry lacks a source link or collection time	App-side validation drops the entry and the draft notes how many were dropped.

Production notes

Do not publish or send automatically. External sources should retain links and collection time. A curation draft may enter the sending pipeline only after editor confirmation; cap the sweep frequency against external sources to avoid load on those sites. Keep long-task run.ids and credentials in your server-side task store — never ship them to the browser.

Next steps

• Read the Quickstart to run the shortest path for Agent identity and delegation.
• Read Authorization and browser sandbox for the security boundaries of controlled sessions.
• Continue with the Social media draft agent for an adjacent scenario.