apps.fas.usda.gov

query-product

Installation

Adds this website's skill for your agents

 

Summary

Query U.S. agricultural import/export/re-export statistics in the USDA FAS Global Agricultural Trade System (GATS) by product (BICO/HS-2/HS-6/HS-10/FAS/FATUS/etc.), year range, destination country or region, and data value type (Value, Unit Value, or Quantity).

FIG. 01
FIG. 02
FIG. 03
FIG. 04
FIG. 05
FIG. 06
FIG. 07
SKILL.md
263 lines

USDA FAS GATS — Query U.S. Agricultural Trade by Product, Year, Destination, Data Value

Purpose

Drive the USDA Foreign Agricultural Service's Global Agricultural Trade System (GATS) "Standard Query" page at apps.fas.usda.gov/gats/ExpressQuery1.aspx to retrieve U.S. agricultural import/export/re-export statistics, scoped by:

  • Product — chosen from a product-group taxonomy (BICO HS-10, FAS, FATUS, Harmonized HS-2/4/6/10, Processed Food, Organics-Selected, WTO HS-6, SSG) or by typing an HS code / commodity name.
  • Year range — annual / monthly / quarterly / two-year series, 1989 (Harmonized) or 1967 (FAS) through the current calendar year.
  • Destination ("Partners") — World Total, a region/partner group (EU-27, ASEAN, China and Hong Kong, etc.), or any of ~250 individual partner countries.
  • Data value type — Value (Dollars / Thousands / Millions / Billions), Unit Value, or Quantity (in MT, FAS Converted, FAS Non Converted, or one of two aggregate UOMs).

Returns a tabular result set inline on the same page. Read-only — no login is required; the page works anonymously.

When to Use

  • "What was the value of U.S. tree-nut exports to Japan from 2020 to 2024?"
  • "Show me U.S. soybean export volume to China by year for the last decade."
  • Bulk pull of HS-10 / HS-6 / HS-2 trade data for an arbitrary partner × product × year window.
  • Anywhere you'd otherwise scrape FATUS / BICO / monthly Census trade releases — GATS is the canonical front-end to those datasets.

Workflow

There are two viable methods for an agent. Use the API path if you have a data.gov key; otherwise drive the Standard Query form in a real browser. There is no anti-bot system on GATS — a vanilla session is sufficient.

Method A — Open Data API (preferred when an api.data.gov X-Api-Key is available)

USDA-FAS publishes a fully documented JSON/XML API for the same dataset GATS displays. Swagger is at https://apps.fas.usda.gov/opendatawebV2/assets/swagger/swagger.json; host is https://api.fas.usda.gov; auth is via the X-Api-Key header issued at https://apps.fas.usda.gov/opendatawebv2/#/signup (free, instant). Without a key the endpoints return 403 from api-umbrella — verified.

Useful endpoints for this skill (all GET, all return JSON when Accept: application/json):

  • /api/gats/regions — region code → name table.
  • /api/gats/countries — partner code (2-char, e.g. JA=Japan, CH=China, MX=Mexico) → name + region.
  • /api/gats/commodities — full HS-10 commodity tree.
  • /api/gats/HS6Commodities — HS-6 sub-tree.
  • /api/gats/unitsOfMeasure — UOM code → label.
  • /api/gats/customsDistricts — U.S. customs district enum.
  • /api/gats/censusExports/partnerCode/{partnerCode}/year/{year}/month/{month} — U.S. exports (per Census) for one partner/year/month. Returns one row per HS-10 commodity with value (USD) and quantity + uom.
  • /api/gats/censusImports/partnerCode/{partnerCode}/year/{year}/month/{month} — same for imports.
  • /api/gats/censusReExports/partnerCode/{partnerCode}/year/{year}/month/{month} — re-exports.
  • /api/gats/customsDistrictExports|Imports|ReExports/partnerCode/{partnerCode}/year/{year}/month/{month} — same but broken down by U.S. district of clearance.
  • /api/gats/UNTradeImports|Exports|ReExports/reporterCode/{reporterCode}/year/{year} — UN ComTrade mirror (annual, no month param).

To answer "product P / year range Y1..Y2 / destination D / value or quantity" via the API:

  1. Fetch /api/gats/countries once, look up the 2-char partnerCode for D.
  2. Fetch /api/gats/commodities once, find the HS code(s) for P (cache both lookups — they change rarely).
  3. For each year in Y1..Y2 and each month 1..12 (or just month 12 for full-year cumulative if available — verify per release calendar), call the matching census{Exports,Imports,ReExports} endpoint, filter the response rows to the HS codes from step 2, and aggregate value / quantity.
  4. Sum / present as the caller requested.

Method B — Standard Query browser form (works without any credential)

This is the path documented in iter-1 of the build — verified live against four distinct query shapes (default BICO aggregates; Japan × Tree Nuts × 2020-2024 × Value; same × Quantity; Japan × HS-2 Chapter 08 × 2020-2024). No proxy or stealth needed.

  1. Open the Standard Query page. Going to https://apps.fas.usda.gov/gats/default.aspx 302s to a JavaScript-driven detectscreen.aspx?returnpage=... that posts the user's screen resolution back to the server before bouncing into the app. Skip the front door and browse open https://apps.fas.usda.gov/gats/ExpressQuery1.aspx directly — it works in one round-trip, no resolution-detect cookie required.

  2. Wait for load + a short settle.

    browse wait load --remote --session "$SID"
browse wait timeout 2500 --remote --session "$SID"

  3. Set form fields via JS eval, then trigger the __doPostBack for each dropdown whose change cascades a server-side rebuild. The form is classic ASP.NET WebForms — every <select> has onchange="__doPostBack('<id>','')" and the server rebuilds dependent fields. browse fill / browse select work but are slow because each cascading postback re-flows the DOM and invalidates the snapshot ref cache; browse eval to set .value + manual __doPostBack is ~4× faster and what's used below.

    All field IDs are under ctl00_ContentPlaceHolder1_*. The full schema:

    Field idForm purposeValues (sample)
    ddlDataSourceData SourceFASUSTR (FAS U.S. Trade — default), USCD (U.S. Customs Districts), STATE (U.S. States)
    ddlProductTypeTrade flowX (Exports — default), M (Imports Consumption), MG (Imports General), RX (Re-Exports)
    ddlProductGroupProduct taxonomyBICO-HS10 (default), BICO-HS6, FAS, FATUS, SSG, OFood, PFood, WTO, HS2, HS4, HS6, HS10
    lb_PartnersMulti-select listbox of destinationsR00=World Total (default), JA=Japan, CH=China, MX=Mexico, KS=South Korea, etc. (full enum returned in HTML)
    lb_ProductsMulti-select listbox of productsDepends on ddlProductGroup. e.g. for BICO: 0195AT=Tree Nuts, 0035AT=Soybeans, 0145AT=Beef & Beef Products; for HS2: 08H2=Chapter 08 (Edible Fruit and Nuts), 12H2=Chapter 12 (Oil Seeds), etc.
    ddlProductSearchSearch byText (name) or Code (HS digits)
    ddlValueTypeValue column type`` (None), GVAL (Value — default), GUVAL (Unit Value)
    ddlValueUnitValue unitsD (Dollars), T (Thousands — default), M (Millions), B (Billions)
    ddlQuantityTypeQuantity column`` (None — default), Q1 (Quantity)
    ddlQuantityUnitQuantity UOMFASN (FAS Non Converted — default once Q1 chosen), FASC (FAS Converted), UOM1, UOM2
    ddlDateSeriesTime aggregationAnnual (default), Monthly, Quarterly, TwoYear
    ddlStartYear / ddlEndYearYear range4-digit year string, e.g. 2020 / 2024. Range 1967–current (BICO/FATUS); 1989–current (Harmonized).
    ddlMYStartMonth / ddlMYEndMonthMonths covered when ddlDateSeries=Monthly01 … 12
    ddlEndMonthCut-off month for current year (YTD column)01 … 12
    ddlPrecedenceRow groupingRPTPTRPRD (Partner/Product — default), RPTPRDPTR (Product/Partner)
    ddlIncludeResult cap00All (default) or 00Top5 / 00Top10 / ... 00Top50
    ddlOrderBySortCODE, DESC, RANK (default)
    ddlInDetailAggregation detailSummary (default), Partner, Product
    ddlAgOnlyFilter to agricultural HS codes only — appears only for non-BICO product groupsYes (default when shown), No
    ddlCalculation% change columnPDPDGROW (Period/Period — default), AVG, AVGGROW, COMPGROW, PDGROW, Sub, `` (None)
    btnRetrieveDataSubmitClick triggers __doPostBack('ctl00$ContentPlaceHolder1$btnRetrieveData','') with ValidationGroup=NewExpressQuery

    Example: U.S. exports of Tree Nuts (BICO product 0195AT) to Japan (JA), 2020-2024, Value in Thousands of USD —

    var p = document.getElementById('ctl00_ContentPlaceHolder1_lb_Partners');
Array.from(p.options).forEach(o => o.selected = (o.value === 'JA'));
var pr = document.getElementById('ctl00_ContentPlaceHolder1_lb_Products');
Array.from(pr.options).forEach(o => o.selected = (o.value === '0195AT'));
document.getElementById('ctl00_ContentPlaceHolder1_ddlStartYear').value = '2020';
document.getElementById('ctl00_ContentPlaceHolder1_ddlEndYear').value = '2024';
__doPostBack('ctl00$ContentPlaceHolder1$btnRetrieveData','');

  4. Wait for the postback + result render.

    browse wait load --remote --session "$SID"
browse wait timeout 4000 --remote --session "$SID"

  5. Read the result grid. The result table renders inline below the form at id ctl00_ContentPlaceHolder1_UltraWebTab1__ctl1_grdExpressQuery_GridView1. Extract via:

    var g = document.getElementById('ctl00_ContentPlaceHolder1_UltraWebTab1__ctl1_grdExpressQuery_GridView1');
var rows = Array.from(g.rows).map(r => Array.from(r.cells).map(c => c.innerText.trim().replace(/\s+/g,' ')));

    Row layout: row 0 = year header, row 1 = column-type header (Value / Qty / UOM), rows 2..N-1 = data rows in [blank, partnerRank, blank, partnerName, productRank, blank, productName, (UOM if Qty), value/qty per year..., periodPctChange], last row = Grand Total.

    The page title stays "Standard Query" after a successful query — don't gate on a URL or title change. Gate on g.rows.length > 2 or on the absence of the "No Data Found..." literal in document.body.innerText.

Site-Specific Gotchas

  • No anti-bot, no login, no proxy. Verified iter-1: a bare browse cloud sessions create (no --verified, no --proxies) reaches GATS just fine. The session this skill was built on used --verified --proxies as a precaution; the proxy was not required. Don't waste budget on stealth here.
  • apps.fas.usda.gov/gats/default.aspx does a JS-driven screen-resolution detect (detectscreen.aspx?returnpage=default.aspx → posts res=WxH&d=N&pw=N back, then bounces). Bare curl/fetch of default.aspx 302s into the JS bouncer and gets stuck. Always start at ExpressQuery1.aspx directly — it does not require the screen-detect to have run.
  • Changing ddlProductGroup clears the lb_Partners selection (and rebuilds lb_Products). Always set ProductGroup first, then Partners and Products. Verified: switching from BICO-HS10 to HS2 silently emptied lb_Partners, causing a follow-up Retrieve to return data scoped to no partner.
  • The "AG Only" dropdown (ddlAgOnly) only appears for non-BICO product groups (Harmonized, Processed Food, WTO, etc.). Default is Yes, which filters to the subset of HS codes USDA marks as agricultural. Set to No to get the full chapter (including any non-ag codes). Won't be present in the DOM when ProductGroup ∈ {BICO-HS10, BICO-HS6, FAS, FATUS, SSG, OFood}.
  • Asterisks (-*) and * markers in the product/partner labels are category-level "not fully agricultural" / "represents a defunct or aggregated entity" markers, not data flags. E.g. "08 - Edible Fruit And Nuts-*" means "Chapter 08 contains some non-ag tariff lines" — pair with ddlAgOnly=No if you want all of it. Belgium-Luxembourg(*) indicates that partner was a single reporting entity before 1999 and now resolves to its dependents listed below (- Belgium(!), - Luxembourg(!)). Strip the trailing markers when emitting clean names.
  • End-year > current calendar year is silently truncated to YTD comparison columns. If you request ddlEndYear=2024 on May 2026, the result columns end up as 2020, 2021, 2022, 2023, Jan - Mar 2023, Jan - Mar 2024 — i.e. the most recent full year is 2023 and 2024 is replaced by a YTD-vs-prior-YTD pair using ddlEndMonth as the cut-off (default March). If the caller wants full annual 2024, the data may simply not exist yet — emit partial: true and surface the YTD pair as the latest two columns rather than mapping it to "2024".
  • Year range upper bound is the current calendar year minus zero (i.e. 2026 is selectable in May 2026) but data for that year is always partial. The two trailing columns after the last full year are always a YTD-Mar pair by default. Adjust ddlEndMonth if you want a longer YTD window.
  • The currently-selected default ddlEndYear value is the present calendar year (e.g. 2026 in May 2026), and ddlStartYear defaults to currentYear - 5. If you don't override these, you'll get the last six years.
  • ddlValueType="" (None) silently disables the Value columns. If a caller asks for "quantity only", set Value to "" and Quantity to Q1. If both are "", the Retrieve produces "No Data Found...".
  • Numeric values in the result grid are formatted with thousands-commas and no currency symbol ("806,765" = 806,765 thousand-USD when ddlValueUnit=T). Parse with parseFloat(s.replace(/,/g,'')). Decimals appear for quantities (244,685.8 MT).
  • Confirmed-blocked / no-go shortcuts — don't waste iterations on these:
    • api.fas.usda.gov/api/gats/* without an X-Api-Key header → 403 (api-umbrella upstream).
    • Plain curl of default.aspx → 302 → detectscreen.aspx → stuck on top.location.href = "detectscreen.aspx?action=set&res=..." JS that requires a real browser. Hit ExpressQuery1.aspx directly.
  • Result page URL never changes from ExpressQuery1.aspx — the entire app is one URL with __VIEWSTATE carrying state. Don't try to deep-link a query via URL params; persist the __VIEWSTATE + __VIEWSTATEGENERATOR + __EVENTVALIDATION form values across postbacks if you want repeatable runs without scripting, or just rebuild from the form fields each time (cheaper, what the skill recommends).
  • There is no success HTTP code distinction between a populated result, a "No Data Found..." result, and a hidden-validator block. Always inspect both g.rows.length and the literal "No Data Found..." in body text before emitting success.
  • Anti-bot footnote — none observed across iter-1. The site issues an ASP.NET_SessionId cookie on first hit; preserve it across postbacks (browse does this for free within one session). Sustained ≤ 1 req/s is well-mannered.

Expected Output

{
  "success": true,
  "query": {
    "trade_flow": "exports",
    "data_source": "FASUSTR",
    "product_group": "BICO-HS10",
    "products": [{ "code": "0195AT", "name": "Tree Nuts" }],
    "partners": [{ "code": "JA", "name": "Japan" }],
    "year_start": 2020,
    "year_end": 2024,
    "date_series": "Annual",
    "value_type": "Value",
    "value_unit": "Thousands of USD",
    "quantity_type": "None"
  },
  "result_url": "https://apps.fas.usda.gov/gats/ExpressQuery1.aspx",
  "columns": ["2020", "2021", "2022", "2023", "2024"],
  "rows": [
    {
      "partner": "Japan",
      "product": "Tree Nuts",
      "values": {
        "2020": 386027,
        "2021": 437162,
        "2022": 401318,
        "2023": 326634,
        "2024": 332508
      },
      "period_pct_change": 2,
      "unit": "Thousands of USD"
    }
  ],
  "grand_total": {
    "2020": 386027, "2021": 437162, "2022": 401318, "2023": 326634, "2024": 332508
  },
  "notes": ["Data Source: U.S. Census Bureau Trade Data", "Product Group: BICO-HS10"],
  "partial": false
}

Alternative shape when quantity_type=Quantity is set (verified Japan × Tree Nuts × 2020-2024):

{
  "success": true,
  "rows": [
    {
      "partner": "Japan",
      "product": "Tree Nuts",
      "quantities": {
        "2020": 57822.8, "2021": 76402.8, "2022": 67652.0,
        "2023": 65923.5, "2024": 64452.3
      },
      "unit": "MT"
    }
  ]
}

Combined Value+Quantity shape when both ddlValueType=GVAL and ddlQuantityType=Q1 (verified Japan × HS-2 Chapter 08 × 2020-2023+YTD — note end-year truncation):

{
  "success": true,
  "partial": true,
  "columns": ["2020", "2021", "2022", "2023", "Jan - Mar 2023", "Jan - Mar 2024"],
  "rows": [{
    "partner": "Japan",
    "product": "08 - Edible Fruit And Nuts",
    "unit": "MT",
    "values":     { "2020": 806765, "2021": 862259, "2022": 749269, "2023": 624802, "Jan - Mar 2023": 168687, "Jan - Mar 2024": 175234 },
    "quantities": { "2020": 244685.8, "2021": 251700.8, "2022": 196353.0, "2023": 176521.3, "Jan - Mar 2023": 56404.0, "Jan - Mar 2024": 53996.9 },
    "period_pct_change_value": 4,
    "period_pct_change_qty": -4
  }]
}

Failure shape when the form returns "No Data Found..." (verified after switching ProductGroup with stale Partner selection):

{
  "success": false,
  "reason": "no_data",
  "hint": "ProductGroup change clears lb_Partners — re-select the destination before submitting."
}