Skip to content
Back to search

telegram-parser

TG is a local-first Telegram comments exporter and analytics dashboard. It exports channel posts, discussion comments, reactions, links, optional media files, and can update existing datasets…

Safety scan

A static text scan, not a guarantee. Select the badge to see what was scanned and every finding.

Checked today

Check it deeper on your machine

It has changed since its scan on 11 Nov 2026.

As a Claude Code plugin, install it and then ask Claude Code:

/plugin marketplace add keithmackay/mackayi/plugin install scanskillsafety@mackayiDo a deep scan of https://github.com/TheMyxa/telegram-parser with scanskillsafety

Or from a terminal, clone it and run it:

git clone https://github.com/keithmackay/scanskillsafety-skill ~/.claude/skills/scanskillsafetynode ~/.claude/skills/scanskillsafety/cli.cjs --deep 'https://github.com/TheMyxa/telegram-parser'

It checks the exact commit: every file, its packages against OSV (api.osv.dev), how it compares with this page's scan (findsafeskills.com receives the repository's address), and, if you agree, Claude reading every file with your own Claude subscription. It never runs the code.

Badge for your README
findsafeskills safety rating

Markdown

[![findsafeskills safety rating](/api/listings/themyxa--telegram-parser/badge.svg)](/listings/themyxa--telegram-parser)

HTML

<a href="/listings/themyxa--telegram-parser"><img src="/api/listings/themyxa--telegram-parser/badge.svg" alt="findsafeskills safety rating" /></a>

Install

i

What was scanned

The files the safety scan read, as written by the repository's author and shown as plain text. Lines with a finding are marked.

README.md · README · 169 lines

View this file on GitHub (read at the repository's current version: the scan did not record which commit it read)

This file has changed since it was scanned, so this is not exactly the text that was scanned, and no line is marked. The findings are listed with the safety scan above.

  1. # TG
  2. [Русский](README_ru.md) | [中文](README_cn.md)
  3. TG is a local-first Telegram comments exporter and analytics dashboard with incremental updates, Watch mode, a scheduler, LLM analysis and an MCP server.
  4. Current version: `3.0.1`.
  5. ## Features
  6. - Export posts, discussion comments, reactions, links and optional media from one or more channels.
  7. - Export to JSON, CSV, Parquet or PostgreSQL; retain compatibility with JSON arrays of posts.
  8. - Refresh older posts during incremental export, reusing downloaded media.
  9. - Batch Watch events and update only changed posts in SQLite.
  10. - Explore charts, activity heatmaps, period comparisons, comment trees, user profiles and global search.
  11. - Run one-off exports, a scheduler or Watch from the dashboard.
  12. - Assign stable readable pseudonyms and restore user identity fields using a private map.
  13. - Analyze JSON exports with configurable ru/en/zh prompts; automate local operations over MCP.
  14. ## Quick start
  15. Install Docker Desktop and start the dashboard:
  16. ```powershell
  17. docker compose up --build dashboard
  18. ```
  19. Open **http://localhost:9595**. Compose creates `.env` from `.env.example` if it is missing. Fill in Telegram settings in the Export tab. For the first Telegram login, run an interactive CLI export and follow the prompts; subsequent runs reuse `sessions/`.
  20. To explore without credentials, upload [examples/example.json](examples/example.json) with **Choose file**. For global search and server-backed views, copy that synthetic file into `data/raw/` and open it as a local export. Uploading alone does not persist or index a file.
  21. ## Screenshots
  22. Real UI captures from version 3.0.1, using only the synthetic example (3 posts, 7 comments). [Reproduce the screenshots](docs/screenshots.md).
  23. ![Dashboard and activity heatmap](docs/screenshots/dashboard.png)
  24. | Post and comment tree | Global comment search |
  25. | --- | --- |
  26. | ![Post and comment tree](docs/screenshots/posts.png) | ![Global comment search](docs/screenshots/search.png) |
  27. ![Export, Scheduler and Watch controls](docs/screenshots/export.png)
  28. ## Configuration
  29. Copy `.env.example` to `.env` to configure it manually. The complete safe template is [.env.example](.env.example).
  30. ```env
  31. API_ID=123456
  32. API_HASH=your_api_hash_here
  33. CHANNEL=your_channel,another_channel
  34. TELEGRAM_SESSION=sessions/session
  35. OUTPUT_FILE=data/raw/export.json
  36. POST_LIMIT=500
  37. INCREMENTAL_LOOKBACK_POSTS=50
  38. POSTS_PAUSE_SECONDS=30
  39. POSTS_PAUSE_AFTER_POSTS=500
  40. # Required only for LLM analysis
  41. LLM_ENDPOINT=http://host.docker.internal:1234/v1/chat/completions
  42. LLM_MODEL=local-model
  43. # Required only for PostgreSQL export
  44. POSTGRES_HOST=host.docker.internal
  45. POSTGRES_PORT=5432
  46. POSTGRES_DB=telegram_parser
  47. POSTGRES_USER=postgres
  48. POSTGRES_PASSWORD=your_postgres_password_here
  49. POSTGRES_TABLE=telegram_comments_export
  50. ```
  51. `export` and `watch` require `API_ID`, `API_HASH` and `CHANNEL`. Only `analyze` requires `LLM_ENDPOINT`/`LLM_MODEL`; PostgreSQL settings are required only for that export target. `CHANNEL` accepts comma-separated names or links. `OUTPUT_FILE` determines the output directory. `POST_LIMIT` limits new/latest posts; `INCREMENTAL_LOOKBACK_POSTS` revisits known older posts. Pause settings control Telegram request pacing. In Docker, `localhost` is the container; use `host.docker.internal` for an LLM/PostgreSQL service on the Docker Desktop host. The project directory is mounted at `/workspace`; configuration uses `/workspace/.env`, while runtime data and sessions are mounted under `/app`.
  52. ## CLI commands
  53. ```powershell
  54. docker compose run --rm cli --help
  55. docker compose run --rm cli config-check
  56. docker compose run --rm cli export json --incremental
  57. docker compose run --rm cli export json --incremental --download-media --anonymize
  58. docker compose run --rm cli export csv
  59. docker compose run --rm cli export parquet
  60. docker compose run --rm cli export postgresql
  61. docker compose run --rm cli watch --poll-interval 30 --refresh-active-posts 20 --anonymize
  62. docker compose run --rm cli index --file examples/example.json
  63. docker compose run --rm cli search "incremental" --file examples/example.json
  64. docker compose run --rm cli compare --from 2026-07-01 --to 2026-08-01 --file examples/example.json
  65. docker compose run --rm cli analyze examples/example.json --limit 3 --language en
  66. docker compose run --rm -i cli mcp
  67. ```
  68. Run locally with `python main.py <command>` after installing `requirements.txt`. The `tg` name shown in help is the parser name, not an installed executable. Docker is preferred when the Windows Python alias is broken. `config-check` validates Telegram, LLM and PostgreSQL settings together; unused optional services can therefore fail that full check.
  69. ## Incremental export, Watch and Scheduler
  70. Incremental datasets are `data/raw/<channel>_dataset.json`, with per-channel JSON state and a derived SQLite index at `data/state/tg.sqlite`. New posts and the configured old-post lookback are merged by `post_id`; comments are merged by `comment_id`.
  71. Watch collects events for one second, deduplicates queued posts and processes up to 20 posts per channel per batch. JSON is read once per batch and atomically rewritten once if changed; SQLite patches only the changed posts, comments, reactions and FTS entries. A missing/stale index triggers a full rebuild. Already fetched changes are saved on FloodWait or task cancellation. The JSON file is still rewritten in full at each changed batch.
  72. Scheduler starts immediately and repeats every `interval_minutes`. If an export is running, a due run is skipped. Scheduler state is in memory: restarting the dashboard stops scheduling. Run history is persisted in SQLite; old `data/scheduler/` JSON records are imported lazily. Watch is a separate long-running process. Avoid concurrent writers to the same dataset/session.
  73. ## Reversible pseudonyms
  74. The default `--anonymize` rules are [anonymization_rules.json](anonymization_rules.json): adjective + animal, for example «Зелёный слон». Names remain stable across Export/Watch restarts, with numeric suffixes on collisions. Scope defaults to a channel; `project` scope is available for a new map. `--anonymizer-file` accepts JSON rules or a legacy text name list.
  75. ```powershell
  76. docker compose run --rm cli anonymize --file examples/example.json --channel demo --output data/raw/demo_anonymous.json
  77. docker compose run --rm cli deanonymize --file data/raw/demo_anonymous.json --channel demo --map data/private/anonymization_map.json --output data/raw/demo_restored.json
  78. ```
  79. Both conversion commands create a new file and refuse to overwrite existing files. `--channel` must match the original channel. The map `data/private/anonymization_map.json` contains original user identities in **unencrypted JSON**; protect it and keep a separate backup. It is excluded from Git, Docker images and HTTP/MCP reads. Text, links and media remain unchanged. Legacy in-memory pseudonyms cannot generally be restored. Incompatible identity modes are rejected during incremental updates. See [pseudonym rules, restoration and migration](docs/anonymization.md).
  80. ## LLM analysis and MCP
  81. Prompt files live in `prompts/llm_ru.json`, `llm_en.json` and `llm_zh.json`. `--prompt-file` overrides `--language`; the JSON object requires `system` and `user_template`, containing `{data}`. Analysis files are written incrementally to `data/analysis/analysis_<source>.json`; SQLite also stores analysis metadata and cache entries.
  82. **Known limitation:** the current `build_post_prompt_payload()` does not return its prepared payload. Verify/fix that before relying on LLM results or content-based cache invalidation. See [implementation notes](docs/architecture.md#known-limitations).
  83. MCP runs over stdio with interactive stdin (`-i`). Tools include `get_config_safe`, `list_exports`, `get_export_summary`, `search_comments`, `search_all_comments`, `get_post`, `list_analysis_files`, `list_analysis_results`, `read_analysis`, `run_analysis`, `start_export`, and `get_export_process_status`. Live export requires `confirm=true`. There is no MCP tool for identity restoration.
  84. ## Errors and recovery
  85. Expected HTTP application failures include `error_code` alongside `ok`/`error`; process conflicts return 409, corrupt data 422, unavailable index 503 and process start failures 500. CLI expected failures return 2 without a traceback; unexpected failures log a traceback and return 1. Invalid export JSON stops processing; the exporter loader saves a backup instead of substituting an empty dataset. Repair/restore the archive before retrying. SQLite failure can use JSON fallback for summary/comparison; programming errors do not silently trigger fallback. [Full error reference](docs/errors.md).
  86. ## Local data and access
  87. Do not commit `.env`, sessions, private identity maps, exports, media, analysis, SQLite state or scheduler history. Keep runtime `.gitkeep` files. The dashboard currently has no authentication and Compose publishes port 9595 on all host interfaces; use it only on a trusted local network or bind `127.0.0.1:9595:9595`. `/api/config` can read/write sensitive settings. Static `/data/` access is restricted to `raw`, `content` and `analysis`; `private`, `state` and `scheduler` are not served.
  88. ## Project structure and developer checks
  89. ```text
  90. main.py CLI entrypoint
  91. api/ HTTP routes and error boundary
  92. services/ Configuration, export queries, processes, scheduler
  93. exporters/ Telegram export and Watch batching
  94. storage/ JSON files, SQLite index, history and LLM cache
  95. telegram/ Message helpers and reversible pseudonyms
  96. analytics/ Summaries and period comparison
  97. llm/ LLM analysis
  98. web/ HTML, CSS and JavaScript dashboard
  99. mcp_server.py MCP tools over stdio
  100. errors.py Expected application errors
  101. version.py Application version
  102. anonymization_rules.json Public pseudonym naming rules
  103. anonymizer Legacy text list of names
  104. prompts/ ru / en / zh prompts
  105. examples/example.json Synthetic demonstration dataset
  106. tests/ Offline regression tests
  107. docs/ Architecture, API, errors, pseudonyms, screenshots
  108. data/raw/ JSON and other exported files
  109. data/content/ Downloaded media
  110. data/analysis/ LLM output
  111. data/state/ SQLite index and per-channel state
  112. data/scheduler/ Legacy JSON scheduler history
  113. data/private/ Private identity map (never publish)
  114. sessions/ Telegram sessions (never publish)
  115. ```
  116. [Architecture](docs/architecture.md) · [HTTP API](docs/api.md) · [Pseudonyms](docs/anonymization.md) · [Errors](docs/errors.md) · [Screenshot workflow](docs/screenshots.md) · [Agent guidance](AGENTS.md)
  117. Use `/workspace` for checks against the current checkout without rebuilding: `/app` contains the image snapshot. These checks do not contact Telegram. Rebuild the dashboard image after changing dependencies or image contents.
  118. ```powershell
  119. docker compose config --quiet
  120. docker compose run --rm --workdir /workspace --entrypoint python cli docs/check_docs.py
  121. docker compose run --rm --workdir /workspace --entrypoint python cli -m unittest discover -s tests -v
  122. docker compose run --rm --workdir /workspace --entrypoint python cli -m compileall -q errors.py main.py mcp_server.py api services exporters storage telegram analytics llm tests
  123. docker compose build dashboard
  124. ```

Ratings and comments

Sign in to rate this skill.

Sign in

Free to use. If it’s helped, pass it on.