Receive "Episode not released!" even though the episode is released on allanime.to

[Lugas7]

Version: 4.9.0
OS: Arch Linux
Shell: bash
Anime: Kami No Tou (Tower of god season 2)

** description **

Get "Episode not released!" when the episode is realeased on allanime.to.

Steps To Reproduce

Happens when I try to watch an episode which is tracked in ani-cli and can thence be selected, but it fails. Think it has something to do with the episode only recently being released (as of my attempt, the episode has only been released 5 hours ago on allanime.to)

Expected behavior
Should be able to watch episode

[HirschBerge]

Also happening on 4.9.0 on NixOS unstable. Has been happening with every episode released in the last 24-36 hours or so that I'm watching. Tower of God, Monogatari, Fairy Tail 100YQ, etc.
Saw something similar a few weeks ago that eventually resolved itself I believe.

[71zenith]

We scrape only a small subset of video providers from allanime. Most of the times when a new episode is released it only has providers we dont scrape.

[Lugas7]

Ah ok, is there an issue for increasing the number of providers scraped? I might be interested in attempting developing that at some point when I have time

[HirschBerge]

We scrape only a small subset of video providers from allanime. Most of the times when a new episode is released it only has providers we dont scrape.

How is this a resolution? Is there an issue/pr to add more providers to scrape this can at least be linked to?

[71zenith]

Ah ok, is there an issue for increasing the number of providers scraped? I might be interested in attempting developing that at some point when I have time

No mostly, as long as the code isnt too complicated.
you can start by looking at the current scrapers

ani-cli/ani-cli

Lines 131 to 197 in 5daa876

	get_links() {
	episode_link="$(curl -e "$allanime_refr" -s "https://${allanime_base}$*" -A "$agent" | sed 's|},{|\
	|g' | sed -nE 's|.*link":"([^"]*)".*"resolutionStr":"([^"]*)".*|\2 >\1|p;s|.*hls","url":"([^"]*)".*"hardsub_lang":"en-US".*|\1|p')"
	case "$episode_link" in
	*vipanicdn* | *anifastcdn*)
	if printf "%s" "$episode_link" | head -1 | grep -q "original.m3u"; then
	printf "%s" "$episode_link"
	else
	extract_link=$(printf "%s" "$episode_link" | head -1 | cut -d'>' -f2)
	relative_link=$(printf "%s" "$extract_link" | sed 's|[^/]*$||')
	curl -e "$allanime_refr" -s "$extract_link" -A "$agent" | sed 's|^#.*x||g; s|,.*|p|g; /^#/d; $!N; s|\
	| >|' | sed "s|>|>${relative_link}|g" | sort -nr
	fi
	;;
	*) [ -n "$episode_link" ] && printf "%s\n" "$episode_link" ;;
	esac
	[ -z "$ANI_CLI_NON_INTERACTIVE" ] && printf "\033[1;32m%s\033[0m Links Fetched\n" "$provider_name" 1>&2
	}
	# innitialises provider_name and provider_id. First argument is the provider name, 2nd is the regex that matches that provider's link
	provider_init() {
	provider_name=$1
	provider_id=$(printf "%s" "$resp" | sed -n "$2" | head -1 | cut -d':' -f2 | sed 's/../&\
	/g' | sed 's/^01$/9/g;s/^08$/0/g;s/^05$/=/g;s/^0a$/2/g;s/^0b$/3/g;s/^0c$/4/g;s/^07$/?/g;s/^00$/8/g;s/^5c$/d/g;s/^0f$/7/g;s/^5e$/f/g;s/^17$/\//g;s/^54$/l/g;s/^09$/1/g;s/^48$/p/g;s/^4f$/w/g;s/^0e$/6/g;s/^5b$/c/g;s/^5d$/e/g;s/^0d$/5/g;s/^53$/k/g;s/^1e$/\&/g;s/^5a$/b/g;s/^59$/a/g;s/^4a$/r/g;s/^4c$/t/g;s/^4e$/v/g;s/^57$/o/g;s/^51$/i/g;' | tr -d '\n' | sed "s/\/clock/\/clock\.json/")
	}
	# generates links based on given provider
	generate_link() {
	case $1 in
	1) provider_init "dropbox" "/Sak :/p" ;; # dropbox(mp4)(single)
	2) provider_init "wetransfer" "/Kir :/p" ;; # wetransfer(mp4)(single)
	3) provider_init "sharepoint" "/S-mp4 :/p" ;; # sharepoint(mp4)(single)
	*) provider_init "gogoanime" "/Luf-mp4 :/p" ;; # gogoanime(m3u8)(multi)
	esac
	[ -n "$provider_id" ] && get_links "$provider_id"
	}
	select_quality() {
	case "$1" in
	best) result=$(printf "%s" "$links" | head -n1) ;;
	worst) result=$(printf "%s" "$links" | grep -E '^[0-9]{3,4}' | tail -n1) ;;
	*) result=$(printf "%s" "$links" | grep -m 1 "$1") ;;
	esac
	[ -z "$result" ] && printf "Specified quality not found, defaulting to best\n" 1>&2 && result=$(printf "%s" "$links" | head -n1)
	printf "%s" "$result" | cut -d'>' -f2
	}
	# gets embed urls, collects direct links into provider files, selects one with desired quality into $episode
	get_episode_url() {
	# get the embed urls of the selected episode
	episode_embed_gql="query (\$showId: String!, \$translationType: VaildTranslationTypeEnumType!, \$episodeString: String!) { episode( showId: \$showId translationType: \$translationType episodeString: \$episodeString ) { episodeString sourceUrls }}"
	resp=$(curl -e "$allanime_refr" -s -G "${allanime_api}/api" --data-urlencode "variables={\"showId\":\"$id\",\"translationType\":\"$mode\",\"episodeString\":\"$ep_no\"}" --data-urlencode "query=$episode_embed_gql" -A "$agent" | tr '{}' '\n' | sed 's|\\u002F|\/|g;s|\\||g' | sed -nE 's|.*sourceUrl":"--([^"]*)".*sourceName":"([^"]*)".*|\2 :\1|p')
	# generate links into sequential files
	cache_dir="$(mktemp -d)"
	providers="1 2 3 4"
	for provider in $providers; do
	generate_link "$provider" >"$cache_dir"/"$provider" &
	done
	wait
	# select the link with matching quality
	links=$(cat "$cache_dir"/* | sed 's|^Mp4-||g;/http/!d;/Alt/d' | sort -g -r -s)
	rm -r "$cache_dir"
	episode=$(select_quality "$quality")
	[ -z "$episode" ] && die "Episode not released!"
	}

[71zenith]

How is this a resolution? Is there an issue/pr to add more providers to scrape this can at least be linked to?

#1410
There is a pr to add back a provider.

[71zenith]

Generally we don't add more cause the current providers are the most stable and don't break unexpectedly

[Lugas7]

One quick improvement would be to change the error message in this event, it is in fact incorrect to say that the episode is not released yet. Perhaps something along the lines of: "episode has not been released in a provider we support" may be better.

I would not have made this issue as a bug if I got a better error message.