Stuck on uploading

[pri-mo]

Hi!
I'm having some issues with uploading with --merge. The tool validates the DB and CSV. Once it starts uploading, it get's stuck on the progress bar (no matter if the CSV file is one record or 6k). Nothing happens for a while and eventually it shows the 429 Client Error (again, no matter the amount of records in csv nor threads I set). Any idea how to proceed?

[vzhd1701]

You have got rate limited by Notion server. Not much you can do here, try using proxy or just wait until the block goes away. In the future runs try reducing --max-threads and don't run it in parallel.

[pri-mo]

I'm sorry to bump this issue but regardless of the length of the CSV I'm importing to merge, I get rejected by the Notion server.

The CSV is two columns (key and relation) and 101 rows (incl. 1 head). This is the reply I get.

INFO: Validating CSV & Notion DB schema
Traceback (most recent call last):
  File "/usr/local/bin/csv2notion", line 8, in <module>
    sys.exit(main())
  File "/usr/local/Cellar/csv2notion/0.3.3/libexec/lib/python3.10/site-packages/csv2notion/cli.py", line 82, in main
    cli(*sys.argv[1:])
  File "/usr/local/Cellar/csv2notion/0.3.3/libexec/lib/python3.10/site-packages/csv2notion/cli.py", line 41, in cli
    notion_rows = convert_csv_to_notion_rows(csv_data, client, collection_id, args)
  File "/usr/local/Cellar/csv2notion/0.3.3/libexec/lib/python3.10/site-packages/csv2notion/cli_steps.py", line 55, in convert_csv_to_notion_rows
    return converter.convert_to_notion_rows(csv_data)
  File "/usr/local/Cellar/csv2notion/0.3.3/libexec/lib/python3.10/site-packages/csv2notion/notion_convert.py", line 43, in convert_to_notion_rows
    notion_rows.append(self._convert_row(row))
  File "/usr/local/Cellar/csv2notion/0.3.3/libexec/lib/python3.10/site-packages/csv2notion/notion_convert.py", line 58, in _convert_row
    columns = self._map_columns(row)
  File "/usr/local/Cellar/csv2notion/0.3.3/libexec/lib/python3.10/site-packages/csv2notion/notion_convert.py", line 84, in _map_columns
    notion_row[col_key] = self._map_column(col_key, col_value, col_type)
  File "/usr/local/Cellar/csv2notion/0.3.3/libexec/lib/python3.10/site-packages/csv2notion/notion_convert.py", line 106, in _map_column
    return conversion_map[value_type](col_value)
  File "/usr/local/Cellar/csv2notion/0.3.3/libexec/lib/python3.10/site-packages/csv2notion/notion_convert.py", line 245, in _map_relation
    resolved_relation = self._resolve_relation_by_key(relation_column, v)
  File "/usr/local/Cellar/csv2notion/0.3.3/libexec/lib/python3.10/site-packages/csv2notion/notion_convert.py", line 258, in _resolve_relation_by_key
    return relation.rows[key]
  File "/usr/local/Cellar/csv2notion/0.3.3/libexec/lib/python3.10/site-packages/csv2notion/notion_db.py", line 46, in rows
    self._cache_rows = self.collection.get_unique_rows()
  File "/usr/local/Cellar/csv2notion/0.3.3/libexec/lib/python3.10/site-packages/csv2notion/notion_db_collection.py", line 21, in get_unique_rows
    sorted_rows = sorted(self.get_rows(), key=lambda r: str(r.title))
  File "/usr/local/Cellar/csv2notion/0.3.3/libexec/lib/python3.10/site-packages/csv2notion/notion_db_collection.py", line 21, in <lambda>
    sorted_rows = sorted(self.get_rows(), key=lambda r: str(r.title))
  File "/usr/local/Cellar/csv2notion/0.3.3/libexec/lib/python3.10/site-packages/notion/maps.py", line 44, in fget
    return api_to_python(self.get(path), **kwargs)
  File "/usr/local/Cellar/csv2notion/0.3.3/libexec/lib/python3.10/site-packages/notion/records.py", line 108, in get
    path, self._get_record_data(force_refresh=force_refresh), default=default
  File "/usr/local/Cellar/csv2notion/0.3.3/libexec/lib/python3.10/site-packages/notion/records.py", line 97, in _get_record_data
    return self._client.get_record_data(
  File "/usr/local/Cellar/csv2notion/0.3.3/libexec/lib/python3.10/site-packages/notion/client.py", line 179, in get_record_data
    return self._store.get(table, id, force_refresh=force_refresh, limit=limit)
  File "/usr/local/Cellar/csv2notion/0.3.3/libexec/lib/python3.10/site-packages/notion/store.py", line 188, in get
    self.call_load_page_chunk(id,limit=limit)
  File "/usr/local/Cellar/csv2notion/0.3.3/libexec/lib/python3.10/site-packages/notion/store.py", line 292, in call_load_page_chunk
    recordmap = self._client.post("loadPageChunk", data).json()["recordMap"]
  File "/usr/local/Cellar/csv2notion/0.3.3/libexec/lib/python3.10/site-packages/notion/client.py", line 282, in post
    response.raise_for_status()
  File "/usr/local/Cellar/csv2notion/0.3.3/libexec/lib/python3.10/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 429 Client Error: Too Many Requests for url: https://www.notion.so/api/v3/loadPageChunk

[vzhd1701]

Please check out new version 0.3.4 and tell how it works for you. It should work OK now without limiting max threads.

[pri-mo]

IT WORKS!!! 🙌🤝🖤🙏🏻

[pablos08]

I'm using 0.3.5 and having a problem that I think is Notion rate limiting. Apparently they only support 3 requests per second

When I use the --merge flag I'm getting constant timeouts and it doesn't work at all anymore. I tried --max-threads 1 but even then I don't get anywhere.

Thanks for making this tool, there's nothing else I've found that can import to Notion and preserve related fields.

[vzhd1701]

@pablos08 could you provide a log with an error that you are getting?