ATTENTION - Anna’s Archive enforces web scraping blocks through Cloudflare. The site is also subject to change. The ebook command may return No Search Results found. A fix is in the works that may involve a new project and archiving this one. A long term solution is hosting an instance of Anna’s Archive with community support or possibly creating a cli by forking the main repo and building an api.
Table of Contents
Installation Methods
PYPI - Stable Release:
-> pipx install getdat
From Head commit of Repository:
-> pipx install git+https://github.com/Audiosutras/getdat.git
Commands
Job
Launches Braintrust, the job board powered by blockchain, in your default browser
Sport
Launches Totalsportk in your default browser
Cinema
Launches movie-web.app in your default browser
-> getdat cinema
Ebook
Search and download an ebook available through Anna’s Archive. You can think of this command as “headless” Anna’s Archive
- The demo for this command downloaded an epub format of Robert Louis Stevenson’s and N.C. Wyeth’s book Treasure Island. This book is in the public domain.
- The demo for this command opens the downloaded ebook using epr, a terminal epub reader to show the contents of the downloaded book. You can expect higher ebook quality by using a desktop e-reader like librum
- Anna’s Archive
SciDBsearch is not yet supported.
-> getdat ebook [OPTIONS] [Q]
ARGUMENTS
| Name | Help |
|---|---|
| Q | Search |
Example:
-> getdat ebook "Treasure Island Stevenson"
or
-> getdat ebook Treasure Island Stevenson
OPTIONS
-> getdat ebook --help
Usage: getdat ebook [OPTIONS] [Q]...
Search and download an ebook available through Anna's Archive
ex: getdat ebook <Search>
Options:
-o, --output_dir TEXT Path to ebook's output directory from home
directory. Path must be prefixed by '~' on
Unix or '~user' on Windows. This argument
overrides GETDAT_BOOK_DIR env var if set.
Outputs book to working directory if neither
are set.
-e, --ext [pdf|epub|mobi|cbr|cbz|fb2|fb2.zip|azw3|djvu]
Preferred ebook extension for ebooks in
search results.
-l, --lang TEXT Preferred language of ebooks in search
results. Provided Language must be ISO 639-1
format. Language region extension supported.
Filtering by multiple languages supported.
Examples: English - en, Spanish - es,
Traditional Chinese - zh-Hant, Multiple
Langauges - en,es,zh-Hant
-i, --instance [org|gs|se] The instance of Anna's Archive you would
like to use for your search: https://annas-
archive.org, https://annas-archive.gs,
https://annas-archive.se - Default: org
--help Show this message and exit.
Example:
-> getdat ebook Treasure Island Stevenson --ext=epub --output_dir=~/books/epub/ --instance=gs --lang=en,es,zh-Hant
or
-> getdat ebook "Treasure Island Stevenson" -e epub -o ~/books/epub -i gs -l es
or
-> getdat ebook "Treasure Island Stevenson"
Environment Variable
GETDAT_BOOK_DIR- Path from home directory to destination directory. Ignored if--output_diris specified as an option
Local Development
Python Version: 3.11. To install python on MacOS & Debian-based systems
-> sudo apt install software-properties-common
-> sudo add-apt-repository ppa:deadsnakes/ppa
-> sudo apt update
-> sudo apt install python3.11
This python package uses poetry for dependency management. To install:
# install pipx if you have not already
-> python3 -m pip install --user pipx
-> python3 -m pipx ensurepath
# install poetry
-> pipx install poetry
Assuming that you have forked the repository and have a copy on your local machine. Within the getdat directory, install dependencies and open a virtualenv shell managed by poetry.
-> poetry install
-> poetry shell
(getdat-py3.11) ->
To run/develop the cli program.
(getdat-py3.11) -> getdat
Usage: getdat [OPTIONS] COMMAND [ARGS]...
A command line utility for getting resources available online
Options:
--help Show this message and exit.
Commands:
cinema Launches movie-web.app in your default browser
ebook Search and download an ebook available through Anna's Archive...
Check out our docs at https://github.com/Audiosutras/getdat/docs for more
details
Project uses pytest for unit testing. Test run quickly with the help of pytest-mock
(getdat-py3.11) -> pytest -v --cov=. tests/
Style guide and code check enforced with pre-commit
(getdat-py3.11) -> pre-commit install
Running docs site locally for making changes to github-pages requires ruby to be installed. After installation make sure to install bundler with gem install jekyll bundler as well:
-> cd docs
-> bundler install
-> bundler exec jekyll build
-> bundler exec jekyll serve
Deployment
Publishing Getdat
Workflow: master branch
-
In
pyproject.tomlbump the version number*.*.* -
Create a git tag with the new version number
*.*.*you specified inpyproject.toml. - Push the newly created tag
git push origin *.*.*to the repository. This will trigger thepre-release.ymlgithub workflow to publish our package totest.pypi. The pre-release can be seen here for testing. Install with:-> python3.11 -m pip install --index-url https://test.pypi.org/simple/ getdat --extra-index-url https://pypi.org/simple beautifulsoup4 requests click- Note:
--extra-index-urloption is pulling dependencies frompypi.organd nottest.pypi.orgthough our package is coming in fromtest.pypi.org. Make sure to add all dependencies from[tool.poetry.dependencies]inpyproject.toml(except python) before running this command.
- Note:
- Create a release on github. Make sure to select
Tagsfrom the toggle menu. Select the latest tag (highest version number). Name the releaseRelease *.*.*. Make sure the version number inpyproject.tomlsyncs up with the release version. ClickPublish release. This will kick off ourrelease.ymlworkflow to publish our package topypi. The release can be seen here and installed for production use. Install with:-> pipx install getdat
Publishing Docs
Docs are published to the site when a release is created for the python package to pypi.org. This is automated, but can be triggered manually by running the github workflow Deploy Jekyll with GitHub Pages dependencies preinstalled.