@@ -365,7 +413,7 @@ extracts only useful data including text, links, pictures and tables,
and returns it as an HTML page with a minimalistic design
optimized for text reading.
txtdot increases the loading speed and reduces client's bandwidth usage
-since no unnecessary code and no scripts are transfered.
+since no unnecessary code and no scripts are transferred.
Also, you won't see any advertisement (unless it's a static picture that is hard to detect as ads).
There are no trackers too.
How to use it
diff --git a/search/search_index.json b/search/search_index.json
index e8188e8..0c4fede 100644
--- a/search/search_index.json
+++ b/search/search_index.json
@@ -1 +1 @@
-{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"Getting Started","text":""},{"location":"#what-is-this","title":"What is this","text":"
txtdot is a proxy that requests the page by the given URL, extracts only useful data including text, links, pictures and tables, and returns it as an HTML page with a minimalistic design optimized for text reading.
txtdot increases the loading speed and reduces client's bandwidth usage since no unnecessary code and no scripts are transfered. Also, you won't see any advertisement (unless it's a static picture that is hard to detect as ads). There are no trackers too.
"},{"location":"#how-to-use-it","title":"How to use it","text":"
txtdot is an open source software, so everyone can host it on his own server. The official instance is txt.dc09.ru, the list of all instances is here.
On the main page, there's a handy form where you can specify a URL, choose an engine and a format for parsed data. On the /get page, \"Home\" button returns you to /, \"Original page\" opens the entered URL in the same window without txtdot proxy.
The latest docs for API endpoints can be found here. For handy JSON API, use /api/parse returning an engine result object (see below). For pure HTML response, use /api/raw-html. Note that both API and browser endpoints on txt.dc09.ru are ratelimited to 2 requests per second.
"},{"location":"#how-it-works","title":"How it works","text":"
This project exists thanks to great Mozilla's Readability.js library. The initial idea was to process HTML with it on the server so the client does not need to download and execute heavy JS, doesn't need to use an adblock.
Readability performs its work very well in most cases. But not always. For example, check any StackOverflow page or Google search results.
So artegoser wrote the basis of the code keeping in mind that we'll extend txtdot with other engines. For now, engines are functions taking a URL as a parameter, returning an object that contains extracted HTML and plain text, page title and language. The object is rendered with ejs template (or, in /api/parse, just sent as JSON).
If an ?engine= parameter wasn't passed, but txtdot found that a specific engine is assigned to the requested domain, for example, \"stackoverflow.com\": stackoverflow, it uses that engine to process the URL. Otherwise, the page is parsed with the engine assigned to * (it's Readability).
"}]}
\ No newline at end of file
+{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"Getting Started","text":""},{"location":"#what-is-this","title":"What is this","text":"
txtdot is a proxy that requests the page by the given URL, extracts only useful data including text, links, pictures and tables, and returns it as an HTML page with a minimalistic design optimized for text reading.
txtdot increases the loading speed and reduces client's bandwidth usage since no unnecessary code and no scripts are transferred. Also, you won't see any advertisement (unless it's a static picture that is hard to detect as ads). There are no trackers too.
"},{"location":"#how-to-use-it","title":"How to use it","text":"
txtdot is an open source software, so everyone can host it on his own server. The official instance is txt.dc09.ru, the list of all instances is here.
On the main page, there's a handy form where you can specify a URL, choose an engine and a format for parsed data. On the /get page, \"Home\" button returns you to /, \"Original page\" opens the entered URL in the same window without txtdot proxy.
The latest docs for API endpoints can be found here. For handy JSON API, use /api/parse returning an engine result object (see below). For pure HTML response, use /api/raw-html. Note that both API and browser endpoints on txt.dc09.ru are ratelimited to 2 requests per second.
"},{"location":"#how-it-works","title":"How it works","text":"
This project exists thanks to great Mozilla's Readability.js library. The initial idea was to process HTML with it on the server so the client does not need to download and execute heavy JS, doesn't need to use an adblock.
Readability performs its work very well in most cases. But not always. For example, check any StackOverflow page or Google search results.
So artegoser wrote the basis of the code keeping in mind that we'll extend txtdot with other engines. For now, engines are functions taking a URL as a parameter, returning an object that contains extracted HTML and plain text, page title and language. The object is rendered with ejs template (or, in /api/parse, just sent as JSON).
If an ?engine= parameter wasn't passed, but txtdot found that a specific engine is assigned to the requested domain, for example, \"stackoverflow.com\": stackoverflow, it uses that engine to process the URL. Otherwise, the page is parsed with the engine assigned to * (it's Readability).
sudo crontab -u txtdot -e\n# The command will open an editor\n# Add this line to the end of the file:\n@reboot sleep 10 && cd /home/txtdot/src && npm run start\n# Save the file and exit\n
Note that built images are not provided via Docker Hub. If you can't or don't want to build them on your server and don't want to setup a CI/CD system, let us know, we'll consider setting up a GitHub Actions workflow.
git clone https://github.com/txtdot/txtdot.git\ncd txtdot\ndocker compose build\ndocker compose up -d\n
sudo crontab -u txtdot -e
+# The command will open an editor
+# Add this line to the end of the file:
+@reboot sleep 10 && cd /home/txtdot/src && npm run start
+# Save the file and exit
+
+
With Docker
+
Docker Engine and Docker Compose are required.
+
Note that built images are not provided via Docker Hub.
+If you can't or don't want to build them on your server
+and don't want to setup a CI/CD system,
+let us know,
+we'll consider setting up a GitHub Actions workflow.