# data.gov.il ingestion recipe (working)

The Israeli Water Authority CKAN API is reachable and usable. Key findings from
getting it to actually respond:

## The one gotcha that matters
`datastore_search` computes a full `COUNT(*)` by default, which **times out** on
these large tables. Always pass **`include_total=false`**. With it, queries
return in well under a second.

## Working endpoints
```
# dataset metadata (fast) -> find resource_id + CSV download url
GET /api/3/action/package_show?id=spring_discharge

# sample rows + schema (fast with include_total=false)
GET /api/3/action/datastore_search?resource_id=<rid>&limit=4&include_total=false

# distinct values of one field (e.g. the spring inventory)
GET /api/3/action/datastore_search?resource_id=<rid>
      &fields=<hebrew col>&distinct=true&limit=500&include_total=false

# one spring's full series, filtered server-side
GET /api/3/action/datastore_search?resource_id=<rid>
      &filters={"שם אנגלית":"AUJA"}&limit=2500&include_total=false
```

## Resource IDs confirmed live
| dataset | resource_id | notes |
|---|---|---|
| spring_discharge (מעיינות – ספיקה מדודה) | `26d650da-9d47-4dd5-8cbe-2f7d6955110b` | ~450 springs, 1960s→2026, L/s |

Spring-discharge schema (columns are Hebrew):
`מספר זיהוי` id · `שם עברית`/`שם אנגלית` name · `שנה הידרולוגית` hydro-year ·
`תאריך מדידת ספיקה` date (dd/mm/yyyy) · `ספיקה (ליטר/שניה)` discharge L/s ·
`שיטת מדידה` method (מד זרם current-meter / סכר weir / הערכה estimate).

## Regional spring picks (from the live inventory)
- **Mountain Aquifer / Judean Desert:** PARA (Ein Prat), AUJA, FAWWAR, DUYUK,
  ARUGOT, DAVID, GIKHON (Jerusalem), QELT AND FAWWAR.
- **Yarkon–Taninim outlet:** `Taninim Spr SUM`.
- **Carmel:** En Carmel, OREN, SUMAQA, ELEK, DALIYYA, SHEFIYYA.
- **Galilee / Golan karst:** DAN Springs SUM, BANIAS, GA'TON springs, TANUR,
  AMMUD (GALIL), MEIRON, the FULIYA/Tabgha group.

## Honest constraint
`web_fetch` returns the JSON into the assistant's context, not to disk, and the
raw 6 MB CSV download comes back empty through it. So bulk ingestion currently
means: query per-spring (small, filtered) and materialize to CSV on this side.
For many springs at scale, a dedicated data connector or a user-provided dump
would be cleaner. The per-spring path (as used for Ein Auja) works fine now.