Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

老哥这么不更新了 #6

Open
qingwaUU opened this issue Dec 4, 2024 · 5 comments
Open

老哥这么不更新了 #6

qingwaUU opened this issue Dec 4, 2024 · 5 comments

Comments

@qingwaUU
Copy link

qingwaUU commented Dec 4, 2024

老哥这么不更新了

@redrainl
Copy link
Owner

redrainl commented Dec 4, 2024

1、因为fofa搜索的组播ip播放流畅度较好,所以自动更新脚本不再从tonkiang获取ip了,改为只从fofa获取。
2、看了一下action的记录,没有更新应该是在获取fofa的搜索结果时,被fofa判定有爬虫嫌疑,限制访问了,所以取不到ip,但我也不知道action用的ip什么规律,不确定哪天的ip能没有限制。
3、前一阵看到部分省市组播没有更新,但没有细看原因,不确定是不是搜索条件需要调整。我自己平常是只用上海本地组播的,脚本也是放在本地跑的,所以需要的话也可以自行修改调整脚本,更加适合自己的需要。

@ykyfy
Copy link

ykyfy commented Dec 12, 2024

1、因为fofa搜索的组播ip播放流畅度较好,所以自动更新脚本不再从tonkiang获取ip了,改为只从fofa获取。 2、看了一下action的记录,没有更新应该是在获取fofa的搜索结果时,被fofa判定有爬虫嫌疑,限制访问了,所以取不到ip,但我也不知道action用的ip什么规律,不确定哪天的ip能没有限制。 3、前一阵看到部分省市组播没有更新,但没有细看原因,不确定是不是搜索条件需要调整。我自己平常是只用上海本地组播的,脚本也是放在本地跑的,所以需要的话也可以自行修改调整脚本,更加适合自己的需要。

大佬 本地怎么跑脚本 打开.sh 提示ip不存在

@redrainl
Copy link
Owner

是类似于“错误:文件 ip/Jiangsu.onlygood.ip 不存在”这样的提示吗?
"ip/${city}.ip"是根据对应选择搜索条件从fofa搜索页面获取的ip
"ip/${city}.onlygood.ip"是对上述ip用nc进行端口检测后初步筛选的活动ip
提示不存在就是没有筛选出活动ip,如果确定应该是有可用搜索结果的,就和手动查询的结果比对一下,看具体问题在哪里

@ykyfy
Copy link

ykyfy commented Dec 24, 2024

是类似于“错误:文件 ip/Jiangsu.onlygood.ip 不存在”这样的提示吗?“ip/${city}.ip”是根据对应选择搜索条件从fofa搜索页面获取的ip “ip/${city}.onlygood.ip”是对上述ip用nc进行端口检测后初步筛选的活动ip prompt不存在就是没有筛选出活动ip,如果确定应该是有可用搜索结果的,就和手动查询的结果比对一下,看具体问题在哪里

就是这个提示 不知道怎么改,我就安装了git 是不是还要安装其它软件

@andyliufeix
Copy link

fofa爬虫可以加上header来规避,

[root@etcd-FA ~]# curl -X GET "https://en.fofa.info/result?qbase64=InVkcHh5IiAmJiBjb3VudHJ5PSJDTiIgJiYgcmVnaW9uPSJTaGFuZ2hhaSIgJiYgb3JnPSJDaGluYSBUZWxlY29tIEdyb3VwIiAmJiBwcm90b2NvbD0iaHR0cCIK"
-H "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,/;q=0.8,application/signed-exchange;v=b3;q=0.7"
-H "Accept-Encoding: gzip, deflate, br, zstd"
-H "Accept-Language: en-GB,en;q=0.9"
-H "Cache-Control: max-age=0"
-H "Connection: keep-alive"
-H "Cookie: isRedirectLang=1; is_mobile=pc; Hm_lvt_4a3f4dd5efb420651f5c2d19cd4b1e6b=1735628022; HMACCOUNT=44F5B6380664CFCF; _ga=GA1.1.586319669.1735628022; befor_router=; baseShowChange=false; viewOneHundredData=false; Hm_lpvt_4a3f4dd5efb420651f5c2d19cd4b1e6b=1735628210; _ga_CX7MDY134G=GS1.1.1735628021.1.1.1735628220.0.0.0"
-H "Host: en.fofa.info"
-H "Expect:"
-H "Referer: https://en.fofa.info/"
-H "Sec-Fetch-Dest: document"
-H "Sec-Fetch-Mode: navigate"
-H "Sec-Fetch-Site: same-origin"
-H "Sec-Fetch-User: ?1"
-H "Upgrade-Insecure-Requests: 1"
-H "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36"
-H "sec-ch-ua: "Chromium";v="130", "Google Chrome";v="130", "Not?A_Brand";v="99""
-H "sec-ch-ua-mobile: ?0"
-H "sec-ch-ua-platform: "Windows"" --output test.html.gz

[root@etcd-FA ~]# gzip -d test.html.gz

完整的Cookie: (__fcd=不知道怎么生成,其他的几个基本可以认为是固定的)
isRedirectLang=1; is_mobile=pc; Hm_lvt_4a3f4dd5efb420651f5c2d19cd4b1e6b=1735628022; HMACCOUNT=628393108BB7EDC3; _ga=GA1.1.586319669.1735628022; befor_router=%2F; baseShowChange=false; viewOneHundredData=false; __fcd=ZCHXCNT6YHZGSD05BCB294ACC63994AB; Hm_lpvt_4a3f4dd5efb420651f5c2d19cd4b1e6b=1735628210; _ga_CX7MDY134G=GS1.1.1735628021.1.1.1735628220.0.0.0

curl --head --include "https://hm.baidu.com/hm.js?4a3f4dd5efb420651f5c2d19cd4b1e6b"
-H "Accept: /"
-H "Accept-Encoding: gzip, deflate, br, zstd"
-H "Accept-Language: en-GB,en;q=0.9"
-H "Connection: keep-alive"
-H "Host: hm.baidu.com"
-H "Referer: https://en.fofa.info/"
-H "Sec-Fetch-Dest: script"
-H "Sec-Fetch-Mode: no-cors"
-H "Sec-Fetch-Site: cross-site"
-H "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36"
-H "sec-ch-ua: "Chromium";v="130", "Google Chrome";v="130", "Not?A_Brand";v="99""
-H "sec-ch-ua-mobile: ?0"
-H "sec-ch-ua-platform: "Windows"" | grep HMACCOUNT

https://static.fofa.info/_nuxt/fofa/8e7aaab.js
t.router.afterEach((function(e, i) {
var r = r || [];
!function() {
var e = t.$cookies.get("user") || {}
, i = t.$cookies.get("fofa_token") && e ? e.mid : "0"
, a = t.$cookies.get("fofa_token") && e ? e.rank_level : "0";
r.push(["_setCustomVar", 1, "uid", i, 1]),
r.push(["_setCustomVar", 2, "level", a, 1]);
var o = "zh" === n.state.targetLang ? "4275507ba9b9ea6b942c7a3f7c66da90" : "4a3f4dd5efb420651f5c2d19cd4b1e6b";
document.getElementById("baidu_tj") && document.getElementById("baidu_tj").remove();
var s = document.createElement("script");
s.src = "https://hm.baidu.com/hm.js?" + o,
s.id = "baidu_tj";
var c = document.getElementsByTagName("script")[0];
c.parentNode.insertBefore(s, c)
}()

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants