Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: shorebird updater operation timed out during preview #2878

Open
nausharipov opened this issue Feb 11, 2025 · 30 comments
Open

fix: shorebird updater operation timed out during preview #2878

nausharipov opened this issue Feb 11, 2025 · 30 comments
Labels
bug Something isn't working

Comments

@nausharipov
Copy link

App ID: c68e2245-86ae-4859-bd1d-6b870ee9173b

Description

Shorebird preview shows an old version because updater operation timed out.

Steps To Reproduce

  1. Follow instructions: https://docs.shorebird.dev/#create-a-release
  2. Make a minor change in your test app
  3. Create a patch
  4. Try to preview it on a connected Android device

Expected Behavior

The patch is downloaded and the minor change is visible.

Additional Context

shorebird preview
✓ Fetching releases (0.8s)
✓ Fetching releases (0.8s)
Which release would you like to preview?
Which release would you like to preview? 0.1.0+1
✓ Fetching aab artifact (0.4s)
✓ Downloading release (0.4s)
✓ Using stable track (0.3s)
✓ Extracting metadata (0.7s)
✓ Built apks: /Users/Darkhan/.shorebird/bin/cache/previews/c68e2245-86ae-4859-bd1d-6b870ee9173b/android_0.1.0+1_697864.apks (3.6s)
✓ Installing apks (18.7s)
✓ Starting app (1.8s)
--------- beginning of main

02-11 13:11:06.822  3939  3939 I flutter : updater::cache::updater_state: [shorebird] No existing state file found: File /data/user/0/com.example.shorebird_test/files/shorebird_updater/state.json does not exist, creating new state.
02-11 13:11:06.823  3939  3939 I flutter : [INFO:flutter/shell/common/shorebird/shorebird.cc(247)] Shorebird updater: no active patch.
02-11 13:11:06.823  3939  3939 I flutter : updater::updater: [shorebird] Reporting launch start.
02-11 13:11:06.824  3939  3939 I flutter : [INFO:flutter/shell/common/shorebird/shorebird.cc(270)] Starting Shorebird update
02-11 13:11:06.827  3939  3990 I flutter : updater::network: [shorebird] Sending patch check request: PatchCheckRequest { app_id: "c68e2245-86ae-4859-bd1d-6b870ee9173b", channel: "stable", release_version: "0.1.0+1", platform: "android", arch: "aarch64" }

--------- beginning of system

02-11 13:11:06.854  3939  3939 I flutter : updater::updater: [shorebird] Reporting successful launch.

02-11 13:11:08.377  3939  3990 I flutter : updater::updater: [shorebird] Patch check response: PatchCheckResponse { patch_available: true, patch: Some(Patch { number: 1, hash: "e9e214da96b2ad8ab0dea21f7a3bb959432c54095a96a4f279659a16d1a89cab", download_url: "https://cdn.shorebird.cloud/api/v1/patches/c68e2245-86ae-4859-bd1d-6b870ee9173b/android/aarch64/156049/dlc.vmcode", hash_signature: None }), rolled_back_patch_numbers: Some([]) }
02-11 13:11:08.378  3939  3990 I flutter : updater::network: [shorebird] Downloading patch from: https://cdn.shorebird.cloud/api/v1/patches/c68e2245-86ae-4859-bd1d-6b870ee9173b/android/aarch64/156049/dlc.vmcode

02-11 13:11:38.395  3939  3990 E flutter : updater::updater: [shorebird] Update failed: error sending request for url (https://cdn.shorebird.cloud/api/v1/patches/c68e2245-86ae-4859-bd1d-6b870ee9173b/android/aarch64/156049/dlc.vmcode)
02-11 13:11:38.395  3939  3990 E flutter : 
02-11 13:11:38.395  3939  3990 E flutter : Caused by:
02-11 13:11:38.395  3939  3990 E flutter :     operation timed out
02-11 13:11:38.397  3939  3990 I flutter : updater::updater: [shorebird] Update thread finished with status: Update had error

@nausharipov nausharipov added the bug Something isn't working label Feb 11, 2025
@bryanoltman
Copy link
Contributor

If the request is timing out, that is likely due to a network connectivity issue on your device. Is it possible your ISP/government is blocking https://cdn.shorebird.cloud? And, just to be sure, your device has internet connectivity?

@nausharipov
Copy link
Author

nausharipov commented Feb 12, 2025

This is strange. I do have internet connection and other services do work fine.
Does this mean Shorebird doesn't work in Kazakhstan?
Is there any workaround for this?
Or maybe your service has wrong configurations that cause it to be blocked?

Image

@eseidel
Copy link
Contributor

eseidel commented Feb 12, 2025

Hmm, we have at least one very large customer in Kazakhstan, so I would expect Shorebird to work. It is concerning that csn.shorebird.cloud is not reachable from your computer.

Does traceroute or curl -I tell you any more? Here is from my machine:

% traceroute cdn.shorebird.cloud         
traceroute: Warning: cdn.shorebird.cloud has multiple addresses; using 104.21.112.1
traceroute to cdn.shorebird.cloud (104.21.112.1), 64 hops max, 40 byte packets
 1  10.45.225.1 (10.45.225.1)  4.363 ms  2.994 ms  3.231 ms
 2  12.63.250.225 (12.63.250.225)  5.482 ms  5.066 ms  5.404 ms
 3  cr2.cgcil.ip.att.net (12.122.99.94)  11.790 ms
    12.122.99.102 (12.122.99.102)  7.505 ms
    cr2.cgcil.ip.att.net (12.122.99.94)  7.213 ms
 4  32.130.17.74 (32.130.17.74)  5.230 ms  6.077 ms
    32.130.17.76 (32.130.17.76)  5.323 ms
 5  32.130.24.205 (32.130.24.205)  5.810 ms  5.997 ms  6.207 ms
 6  12.116.153.210 (12.116.153.210)  20.494 ms  6.056 ms *
 7  141.101.73.202 (141.101.73.202)  7.437 ms
    141.101.73.206 (141.101.73.206)  11.638 ms
    141.101.73.222 (141.101.73.222)  5.074 ms
 8  104.21.112.1 (104.21.112.1)  6.031 ms  5.771 ms  5.575 ms

% curl -I cdn.shorebird.cloud
HTTP/1.1 301 Moved Permanently
Date: Wed, 12 Feb 2025 16:11:00 GMT
Content-Type: text/html
Content-Length: 167
Connection: keep-alive
Cache-Control: max-age=3600
Expires: Wed, 12 Feb 2025 17:11:00 GMT
Location: https://cdn.shorebird.cloud/
Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=BKOXTeO5zha%2FWCwl72iTxDiZ6RewtwOHiGagV9HYWUzvCJMpzYLtDyG%2BcVcoR68SQYaMZiUT6Do3PL9obq7OQi78xfxgZ9dPz33L%2FcJqL%2FvDumMwt8phTr0xZ7ORgzGJK%2BRbPBiu"}],"group":"cf-nel","max_age":604800}
NEL: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
Server: cloudflare
CF-RAY: 910dda209aacced8-ORD
alt-svc: h3=":443"; ma=86400
server-timing: cfL4;desc="?proto=TCP&rtt=5729&min_rtt=5729&rtt_var=2864&sent=1&recv=3&lost=0&retrans=0&sent_bytes=0&recv_bytes=83&delivery_rate=0&cwnd=222&unsent_bytes=0&cid=0000000000000000&ts=0&x=0"

@nausharipov
Copy link
Author

nausharipov commented Feb 12, 2025

I tried with 2 different ISPs and with cellular data, without using VPN.

traceroute cdn.shorebird.cloud
traceroute: Warning: cdn.shorebird.cloud has multiple addresses; using 104.21.112.1
traceroute to cdn.shorebird.cloud (104.21.112.1), 64 hops max, 40 byte packets
 1  192.168.68.1 (192.168.68.1)  3.961 ms  3.492 ms  5.502 ms
 2  smk-c01-br03.2day.kz (80.241.35.52)  6.275 ms  5.354 ms  5.479 ms
 3  172.26.46.1 (172.26.46.1)  8.585 ms  8.478 ms  7.376 ms
 4  172.26.46.102 (172.26.46.102)  5.008 ms  4.780 ms *
 5  172.26.46.62 (172.26.46.62)  10.255 ms
    172.26.46.46 (172.26.46.46)  6.176 ms  9.463 ms
 6  172.26.46.49 (172.26.46.49)  5.029 ms  5.908 ms  5.216 ms
 7  172.26.46.109 (172.26.46.109)  8.659 ms  7.420 ms  7.846 ms
 8  * * *
 9  * * *
10  * * *
11  * * *
12  * * *
13  * * *
14  * * *
15  * * *
16  * * *
17  * * *
18  * * *
19  * *

curl -I cdn.shorebird.cloud returns nothing:

Image

@eseidel
Copy link
Contributor

eseidel commented Feb 12, 2025

I'm not sure what to say. It looks like the packets are routing through 2day.kz (which I presume is your ISP?) and then never successfully coming back out into cloudflare or other addresses that respond to traceroute packets.

We're certainly not trying to block kazakhstan and I don't see any reported outages related to kazakhstan on Cloudflare, so I'm not quite sure where the disruption is coming from.

https://radar.cloudflare.com/traffic/kz

@eseidel
Copy link
Contributor

eseidel commented Feb 12, 2025

@nausharipov
Copy link
Author

The ISP I'm using is called Beeline. I'll contact them tomorrow. Another ISP I tried is called Telecom Community.

@eseidel
Copy link
Contributor

eseidel commented Feb 12, 2025

The community post to cloudflare suggested that sometimes ISPs block all of cloudflare (or a range of cloudflare ip addresses) when trying to block a single (unrelated) site.

You could also use dig to resolve cdn.shorebird.cloud to IPs and then try individual IPs. I expect it will resolve to different ips on a rotating basis, and it's possible that some of them will work for you and some may be blocked by your ISP if it is blocking a range of cloudflare addresses.

dig cdn.shorebird.cloud 

; <<>> DiG 9.10.6 <<>> cdn.shorebird.cloud
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 30417
;; flags: qr rd ra; QUERY: 1, ANSWER: 7, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;cdn.shorebird.cloud.           IN      A

;; ANSWER SECTION:
cdn.shorebird.cloud.    300     IN      A       104.21.112.1
cdn.shorebird.cloud.    300     IN      A       104.21.80.1
cdn.shorebird.cloud.    300     IN      A       104.21.96.1
cdn.shorebird.cloud.    300     IN      A       104.21.32.1
cdn.shorebird.cloud.    300     IN      A       104.21.64.1
cdn.shorebird.cloud.    300     IN      A       104.21.16.1
cdn.shorebird.cloud.    300     IN      A       104.21.48.1

;; Query time: 57 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Wed Feb 12 16:02:44 CST 2025
;; MSG SIZE  rcvd: 160

@nausharipov
Copy link
Author

nausharipov commented Feb 13, 2025

% dig cdn.shorebird.cloud 

; <<>> DiG 9.10.6 <<>> cdn.shorebird.cloud
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 17287
;; flags: qr rd ra; QUERY: 1, ANSWER: 7, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;cdn.shorebird.cloud.		IN	A

;; ANSWER SECTION:
cdn.shorebird.cloud.	300	IN	A	104.21.112.1
cdn.shorebird.cloud.	300	IN	A	104.21.64.1
cdn.shorebird.cloud.	300	IN	A	104.21.16.1
cdn.shorebird.cloud.	300	IN	A	104.21.48.1
cdn.shorebird.cloud.	300	IN	A	104.21.96.1
cdn.shorebird.cloud.	300	IN	A	104.21.80.1
cdn.shorebird.cloud.	300	IN	A	104.21.32.1

;; Query time: 156 msec
;; SERVER: 5.34.34.5#53(5.34.34.5)
;; WHEN: Thu Feb 13 16:26:36 +05 2025
;; MSG SIZE  rcvd: 160

@eseidel
Copy link
Contributor

eseidel commented Feb 13, 2025

Are any of those ip addresses reachable via ping or traceroute? e.g. for me:

% traceroute 104.21.112.1 
traceroute to 104.21.112.1 (104.21.112.1), 64 hops max, 40 byte packets
 1  10.45.225.1 (10.45.225.1)  3.746 ms  2.765 ms  3.110 ms
 2  12.63.250.225 (12.63.250.225)  5.118 ms  5.029 ms  5.253 ms
 3  cr2.cgcil.ip.att.net (12.122.99.94)  8.720 ms
    12.122.99.102 (12.122.99.102)  13.161 ms  5.977 ms
 4  32.130.17.76 (32.130.17.76)  5.090 ms
    32.130.17.74 (32.130.17.74)  5.721 ms
    32.130.17.76 (32.130.17.76)  4.988 ms
 5  32.130.24.203 (32.130.24.203)  5.630 ms
    32.130.24.205 (32.130.24.205)  5.527 ms
    32.130.24.203 (32.130.24.203)  5.352 ms
 6  12.116.153.210 (12.116.153.210)  6.683 ms  6.572 ms  6.420 ms
 7  141.101.73.206 (141.101.73.206)  18.108 ms
    141.101.73.76 (141.101.73.76)  7.162 ms
    141.101.73.218 (141.101.73.218)  5.037 ms
 8  104.21.112.1 (104.21.112.1)  6.669 ms  4.996 ms  6.186 ms

@nausharipov
Copy link
Author

Here are the first 3 IPs from my dig cdn.shorebird.cloud:

% traceroute 104.21.112.1
traceroute to 104.21.112.1 (104.21.112.1), 64 hops max, 40 byte packets
 1  192.168.68.1 (192.168.68.1)  23.869 ms  7.035 ms  9.051 ms
 2  jzk-c01-br04.2day.kz (80.241.35.6)  12.811 ms  8.471 ms  14.034 ms
 3  172.26.46.1 (172.26.46.1)  12.213 ms  12.828 ms  13.755 ms
 4  * * *
 5  172.26.46.59 (172.26.46.59)  6.612 ms
    172.26.46.43 (172.26.46.43)  7.288 ms  8.355 ms
 6  172.26.46.33 (172.26.46.33)  31.664 ms  6.486 ms *
 7  172.26.46.97 (172.26.46.97)  17.278 ms  9.211 ms  8.833 ms
 8  * * *
 9  * * *
10  *
% traceroute 104.21.64.1
traceroute to 104.21.64.1 (104.21.64.1), 64 hops max, 40 byte packets
 1  192.168.68.1 (192.168.68.1)  8.530 ms  4.778 ms  7.871 ms
 2  jzk-c01-br04.2day.kz (80.241.35.6)  7.104 ms  7.035 ms  6.371 ms
 3  172.26.46.1 (172.26.46.1)  10.616 ms  9.034 ms  10.510 ms
 4  172.26.46.102 (172.26.46.102)  7.679 ms * *
 5  172.26.46.43 (172.26.46.43)  6.282 ms
    172.26.46.59 (172.26.46.59)  6.529 ms  8.015 ms
 6  172.26.46.49 (172.26.46.49)  8.295 ms  7.013 ms  7.158 ms
 7  172.26.46.109 (172.26.46.109)  8.368 ms  11.262 ms  11.513 ms
 8  * * *
 9  * *
traceroute 104.21.16.1
traceroute to 104.21.16.1 (104.21.16.1), 64 hops max, 40 byte packets
 1  192.168.68.1 (192.168.68.1)  13.787 ms  5.843 ms  6.730 ms
 2  jzk-c01-br04.2day.kz (80.241.35.6)  7.268 ms  8.932 ms  9.857 ms
 3  172.26.46.1 (172.26.46.1)  9.470 ms  11.208 ms  9.389 ms
 4  * * *
 5  172.26.46.43 (172.26.46.43)  7.562 ms
    172.26.46.59 (172.26.46.59)  8.623 ms  10.042 ms
 6  172.26.46.49 (172.26.46.49)  8.560 ms  8.892 ms  11.660 ms
 7  172.26.46.109 (172.26.46.109)  11.367 ms  9.079 ms  10.403 ms
 8  85.29.165.225 (85.29.165.225)  10.741 ms  10.899 ms  7.261 ms
 9  * client.fttb.2day.kz (176.222.188.233)  30.121 ms  29.391 ms
10  client.fttb.2day.kz (176.222.188.234)  24.127 ms  22.573 ms  23.353 ms
11  104.21.16.1 (104.21.16.1)  22.615 ms  26.567 ms  22.875 ms

@eseidel
Copy link
Contributor

eseidel commented Feb 14, 2025

I'm not sure. That would suggest that you can reach the 3rd IP but not the first two. Which would mean that either cdn.shorebird.cloud is advertising the wrong IPs in your region, or something is blocking your connection to the first two.

It's possible that cdn.shorebird.cloud would resolve to a different set of IPs from a different computer (e.g. that your computer is caching the resolution). You could try using a different DNS server e.g. 1.1.1.1 (cloudflare) or 8.8.8.8 (google)

cdn.shorebird.cloud is hosted by cloudflare and the dns entries managed by cloudflare (dynamically per region I suspect).

You could compare:
dig @1.1.1.1 shorebird.dev A +short
dig @8.8.8.8 shorebird.dev A +short
dig shorebird.dev A +short

Will show you the DNS response from Cloudflare's DNS host vs. Google's if you run it without an explicit "@host" it will just use your local machine/network DNS setup.

e.g. on my machine:

eseidel@mac ~ % dig @1.1.1.1 shorebird.dev A +short
185.199.109.153
185.199.110.153
185.199.111.153
185.199.108.153
eseidel@mac ~ % dig @8.8.8.8 shorebird.dev A +short
185.199.109.153
185.199.108.153
185.199.111.153
185.199.110.153
eseidel@mac ~ % dig shorebird.dev A +short
185.199.110.153
185.199.109.153
185.199.108.153
185.199.111.153

That would tell you if somehow you're DNS cache is stale (and thus trying to connect to stale IPs is causing the issue). If they all 3 say the same IPs and they're all 3 providing partially-unreachable IPs, i would expect that the issue is with your ISP.

Here are docs on dig if you're curious: https://www.ibm.com/docs/en/aix/7.1?topic=d-dig-command (it's just a command line DNS client useful for debugging possibly DNS-issues).

@nausharipov
Copy link
Author

They are all the same IPs:

% dig @1.1.1.1 shorebird.dev A +short
185.199.108.153
185.199.111.153
185.199.109.153
185.199.110.153
% dig @8.8.8.8 shorebird.dev A +short
185.199.109.153
185.199.108.153
185.199.111.153
185.199.110.153
dig shorebird.dev A +short
185.199.111.153
185.199.110.153
185.199.109.153
185.199.108.153

Checking availability of some of them:

% traceroute 185.199.108.153
traceroute to 185.199.108.153 (185.199.108.153), 64 hops max, 40 byte packets
 1  192.168.68.1 (192.168.68.1)  15.380 ms  6.899 ms  4.934 ms
 2  ast-c01-br03.2day.kz (80.241.35.13)  8.066 ms  6.869 ms  7.624 ms
 3  172.26.46.1 (172.26.46.1)  8.367 ms  9.809 ms  10.256 ms
 4  172.26.46.102 (172.26.46.102)  7.481 ms * *
 5  172.26.46.43 (172.26.46.43)  17.104 ms
    172.26.46.59 (172.26.46.59)  8.166 ms  8.037 ms
 6  172.26.46.49 (172.26.46.49)  7.244 ms *  16.382 ms
 7  172.26.46.109 (172.26.46.109)  10.444 ms  10.146 ms  8.330 ms
 8  * 85.29.165.225 (85.29.165.225)  17.465 ms  8.783 ms
 9  * * *
10  * 172.23.251.163 (172.23.251.163)  41.304 ms  34.086 ms
11  178.210.33.68 (178.210.33.68)  54.337 ms  51.594 ms  51.956 ms
12  5.187.73.201 (5.187.73.201)  89.679 ms
    5.187.73.33 (5.187.73.33)  87.368 ms  89.019 ms
13  * * *
14  be2340.rcr22.fra06.atlas.cogentco.com (154.25.14.233)  96.297 ms  92.629 ms  90.868 ms
15  * * *
16  * * *
17  * * *
18  * *
% traceroute 185.199.109.153
traceroute to 185.199.109.153 (185.199.109.153), 64 hops max, 40 byte packets
 1  192.168.68.1 (192.168.68.1)  16.755 ms  6.446 ms  6.110 ms
 2  ast-c01-br03.2day.kz (80.241.35.13)  9.182 ms  4.787 ms  5.736 ms
 3  172.26.46.1 (172.26.46.1)  8.699 ms  10.964 ms  7.705 ms
 4  * * *
 5  172.26.46.43 (172.26.46.43)  14.798 ms
    172.26.46.59 (172.26.46.59)  6.890 ms  5.878 ms
 6  172.26.46.49 (172.26.46.49)  6.553 ms  6.010 ms  6.985 ms
 7  172.26.46.109 (172.26.46.109)  9.012 ms  9.418 ms  7.287 ms
 8  * 85.29.165.225 (85.29.165.225)  9.871 ms  10.467 ms
 9  * * *
10  * 172.23.251.163 (172.23.251.163)  42.718 ms  35.689 ms
11  178.210.33.68 (178.210.33.68)  51.751 ms  51.952 ms  51.869 ms
12  5.187.73.33 (5.187.73.33)  87.586 ms  86.681 ms  86.837 ms
13  * * *
14  be2340.rcr22.fra06.atlas.cogentco.com (154.25.14.233)  98.492 ms  92.658 ms  88.218 ms
15  * * *
16  * * *
17  * *
% traceroute 185.199.111.153
traceroute to 185.199.111.153 (185.199.111.153), 64 hops max, 40 byte packets
 1  192.168.68.1 (192.168.68.1)  14.847 ms  6.452 ms  7.780 ms
 2  ast-c01-br03.2day.kz (80.241.35.13)  5.840 ms  6.645 ms  7.306 ms
 3  172.26.46.1 (172.26.46.1)  9.516 ms  9.326 ms  9.745 ms
 4  172.26.46.102 (172.26.46.102)  7.320 ms * *
 5  172.26.46.59 (172.26.46.59)  11.332 ms
    172.26.46.43 (172.26.46.43)  5.611 ms  8.044 ms
 6  172.26.46.33 (172.26.46.33)  8.140 ms  7.459 ms *
 7  172.26.46.97 (172.26.46.97)  29.007 ms  13.188 ms  10.260 ms
 8  * 85.29.165.225 (85.29.165.225)  10.828 ms  11.646 ms
 9  * * *
10  * * *
11  178.210.33.68 (178.210.33.68)  61.284 ms  50.574 ms  54.129 ms
12  5.187.73.33 (5.187.73.33)  86.271 ms  88.743 ms  85.270 ms
13  * * *
14  be2340.rcr22.fra06.atlas.cogentco.com (154.25.14.233)  97.918 ms  88.866 ms  87.473 ms
15  * * *
16  * * *
17  * * *
18  * * *
19  * *

@eseidel
Copy link
Contributor

eseidel commented Feb 14, 2025

https://www.cloudflarestatus.com/ lists kazakhstan as "rerouted" and "partially rerouted", but I don't know what it's trying to communicate with those terms.

I think the short answer is that we'd love to help, but don't currently have any alternative other than cloudflare, so if our cloudflare address is not reachable from your machine, unfortunately it won't be possible to preview from that machine.

@nausharipov
Copy link
Author

nausharipov commented Feb 14, 2025

https://cdn.shorebird.dev/ is unreachable from my iPhone using cellular data (Safari and Brave browsers). Are users from Kazakhstan getting their app updates from Shorebird without issues?

image

@eseidel
Copy link
Contributor

eseidel commented Feb 14, 2025

Sorry, that's my typo. It's cdn.shorebird.cloud (which should give you a 400 error) rather than cdn.shorebird.dev (which doesn't exist). cdn.shorebird.cloud is just the URL that is used to vend the cloud flare caching of patch artifacts.

@nausharipov
Copy link
Author

My team is considering to use code push in our Flutter app release process. Our app is one of the most popular in finance category in Kazakhstan.

Are users from KZ getting their updates from Shorebird without issues?
If so, is there a workaround my team can use for this issue? Changing ISP doesn't seem promising, because all ISPs in Kazakhstan depend on one common service.

@eseidel
Copy link
Contributor

eseidel commented Feb 14, 2025

We've not done anything for our service that is specific to Kazakhstan. So what we're seeing here is seems likely to be cloudflare related. Cloudflare accounts for a sizable percentage of all internet traffic, so I would expect issues with cloudflare to be widespread though.

From our side, the only options I can think of are to reach out to cloudflare, change providers, or wait. Maybe we could also see if we could try sharing a different cloudflare url, in case our specific URL was being blocked or is somehow misconfigured. 🤔

It seems unlikely we will be able to address any of these soon however. We're busy with some non-networking work at the moment.

@nausharipov
Copy link
Author

nausharipov commented Feb 14, 2025

Ok. Thank you. It was an honor to investigate with you. Hopefully, we'll be able to use Shorebird soon.

@eq-devs
Copy link

eq-devs commented Feb 27, 2025

Hello, I also want to use CodePush, but I have encountered the same problem recently. I hope it will be solved soon. Thanks to the Shorebird team for their selfless dedication.

@nausharipov
Copy link
Author

@eq-devs what country are you located in? Is it the exact same error?

@urmatt
Copy link

urmatt commented Feb 27, 2025

Hello! I also having problems with requesting https://cdn.shorebird.cloud url.

updater::updater: [shorebird] Update failed: error sending request for url (https://cdn.shorebird.cloud/api/v1/patches/123123
Caused by:
                  operation timed out

My country is Kyrgyz Republic.
Network mobile operator "O!"

Update:
After reconnect to network it works.

@eq-devs
Copy link

eq-devs commented Feb 27, 2025

@eq-devs what country are you located in? Is it the exact same error?

from Kazakhstan

@nausharipov
Copy link
Author

@bryanoltman this appears to be a widespread and critical issue in our region.

@eseidel
Copy link
Contributor

eseidel commented Mar 3, 2025

Bryan is currently out of the office (and Felix and I are unfortunately quite busy with other issues atm).

If I'm understanding correctly what is happening is that Kazakhstan ISP(s) are blocking some Cloudflare IPs, but not all so

  • cdn.shorebird.cloud is not reliably reachable, however
  • api.shorebird.dev does appear to be reliably reachable.
    Correct?

Is there a site we can use for testing Kazakhstan reachability? e.g. https://ping.chinaz.com/cdn.shorebird.cloud is a tool we use for testing site-reachability from within China.

@eq-devs
Copy link

eq-devs commented Mar 3, 2025

Bryan is currently out of the office (and Felix and I are unfortunately quite busy with other issues atm).

If I'm understanding correctly what is happening is that Kazakhstan ISP(s) are blocking some Cloudflare IPs, but not all so

  • cdn.shorebird.cloud is not reliably reachable, however
  • api.shorebird.dev does appear to be reliably reachable.
    Correct?

Is there a site we can use for testing Kazakhstan reachability? e.g. https://ping.chinaz.com/cdn.shorebird.cloud is a tool we use for testing site-reachability from within China.

Whitelabel Error Page
This application has no explicit mapping for /error, so you are seeing this as a fallback.

Tue Mar 04 02:03:33 CST 2025
There was an unexpected error (type=Internal Server Error, status=500).
An error happened during template parsing (template: "class path resource [templates/mping.html]")

The above is the result of testing with this tool.
It seems that our operator has indeed blocked shorebird cdn. Waiting for the update🆕

@nausharipov
Copy link
Author

@urmatt is in Kyrgyzstan and they have the same issue, @eseidel

These services indicate that cdn.shorebird.cloud is unavailable in KZ and Russia:
https://kazbt.com/
https://ping-admin.ru/

@eseidel
Copy link
Contributor

eseidel commented Mar 3, 2025

My guess is that the operator has not intentionally blocked Shorebird CDN, so much as they've blocked a range of Cloudflare IP addresses, and our service happens to use some of those same entry-points. Cloudflare and Google (do not assign unique addresses for each service using them, rather they have a pool of addresses they use for most/all services). In attempting to block some other service that uses Cloudflare, the operator has also ended up blocking a variety of other services, including ours.

Probably what we eventually need to do is to teach our tooling how to fallback to a separate CDN (e.g. our google services directly) if cloudflare is unreachable.

@urmatt
Copy link

urmatt commented Mar 5, 2025

My guess is that the operator has not intentionally blocked Shorebird CDN, so much as they've blocked a range of Cloudflare IP addresses, and our service happens to use some of those same entry-points. Cloudflare and Google (do not assign unique addresses for each service using them, rather they have a pool of addresses they use for most/all services). In attempting to block some other service that uses Cloudflare, the operator has also ended up blocking a variety of other services, including ours.

Probably what we eventually need to do is to teach our tooling how to fallback to a separate CDN (e.g. our google services directly) if cloudflare is unreachable.

Yes we need it

@nausharipov
Copy link
Author

Does this mean that all users of apps with shorebird are not getting updates if they are located in Kazakhstan, Kyrgyzstan and Russia?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

5 participants