| Port details |
- py-dns-crawler Crawler for getting info about DNS domains and services
- 1.6.5 dns
 =2      1.6.3_1Version of this port present on the latest quarterly branch. - Maintainer: sunpoet@FreeBSD.org
- Port Added: 2021-06-14 17:48:47
- Last Update: 2024-11-27 02:34:19
- Commit Hash: 6d647cb
- People watching this port, also watch:: jdictionary, py311-Automat, py311-python-gdsii, py39-PyOpenGL, p5-Sane
- Also Listed In: python
- License: GPLv3
- WWW:
- https://gitlab.nic.cz/adam/dns-crawler
- Description:
- Despite the name, the crawler gets info for more services than just DNS:
- DNS:
- all A/AAAA records (for the 2nd level domain and www.subdomain), annotated
with GeoIP
- TXT records (with SPF and DMARC parsed for easier filtering)
- TLSA (for the 2nd level domain and www.subdomain)
- MX
- DNSSEC validation
- nameservers:
- each server IP annotated with GeoIP
- HOSTNAME.BIND, VERSION.BIND, AUTHORS.BIND and fortune (also for all IPs)
- users can add custom additional RRs in the config file
- E-mail (for every server from MX):
- SMTP server banners (optional, ports are configurable)
- TLSA records
- Web:
- HTTP status & headers (inc. parsed cookies) for ports 80 & 443 on each IP
from A/AAAA records
- certificate info for HTTPS (optionally with an entire cert chain)
- webpage content (optional)
- everything of the above is saved for each step in the redirect history --
the crawler follows redirects until it gets a non-redirecting status or hits
a configurable limit
  ¦  ¦  ¦  ¦ 
- Manual pages:
- FreshPorts has no man page information for this port.
- pkg-plist: as obtained via:
make generate-plist - There is no configure plist information for this port.
- Dependency lines:
-
- ${PYTHON_PKGNAMEPREFIX}dns-crawler>0:dns/py-dns-crawler@${PY_FLAVOR}
- To install the port:
- cd /usr/ports/dns/py-dns-crawler/ && make install clean
- To add the package, run one of these commands:
- pkg install dns/py-dns-crawler
- pkg install py311-dns-crawler
NOTE: If this package has multiple flavors (see below), then use one of them instead of the name specified above.
NOTE: This is a Python port. Instead of py311-dns-crawler listed in the above command, you can pick from the names under the Packages section.- PKGNAME: py311-dns-crawler
- Package flavors (<flavor>: <package>)
- distinfo:
- TIMESTAMP = 1732260944
SHA256 (dns_crawler-1.6.5.tar.gz) = f4ee4e98b7fae84e71e51422736ff1cdcf3f01653b5c6f73cb190f79b46115e4
SIZE (dns_crawler-1.6.5.tar.gz) = 45203
Packages (timestamps in pop-ups are UTC):
- Dependencies
- NOTE: FreshPorts displays only information on required and default dependencies. Optional dependencies are not covered.
- Build dependencies:
-
- py311-setuptools>=41 : devel/py-setuptools@py311
- py311-setuptools-git-versioning>=0 : devel/py-setuptools-git-versioning@py311
- py311-wheel>=0 : devel/py-wheel@py311
- python3.11 : lang/python311
- py311-build>=0 : devel/py-build@py311
- py311-installer>=0 : devel/py-installer@py311
- Test dependencies:
-
- python3.11 : lang/python311
- Runtime dependencies:
-
- py311-asn1crypto>=1.5.1 : devel/py-asn1crypto@py311
- py311-cert-human>=1.0.7 : security/py-cert-human@py311
- py311-checkdmarc>=4.3.1 : mail/py-checkdmarc@py311
- py311-dnspython>=2.3.0,1 : dns/py-dnspython@py311
- py311-ecdsa>=0.18.0 : security/py-ecdsa@py311
- py311-forcediphttpsadapter>=1.0.2 : www/py-forcediphttpsadapter@py311
- py311-GeoIP2>=3.0.0 : net/py-GeoIP2@py311
- py311-hstspreload>=0 : www/py-hstspreload@py311
- py311-idna>=2.10 : dns/py-idna@py311
- py311-pyaml>=21.10.1 : textproc/py-pyaml@py311
- py311-pyicu>=2.13.1 : devel/py-pyicu@py311
- py311-openssl>=23.1.1,1 : security/py-openssl@py311
- py311-python-magic>=0.4.27 : devel/py-python-magic@py311
- py311-redis>=4.5.4 : databases/py-redis@py311
- py311-requests>=2.23.0 : www/py-requests@py311
- py311-requests-toolbelt>=0.10.1 : www/py-requests-toolbelt@py311
- py311-rq1>=1.13.0 : devel/py-rq1@py311
- py311-cryptography>=42.0.8,1 : security/py-cryptography@py311
- python3.11 : lang/python311
- There are no ports dependent upon this port
Configuration Options:- No options to configure
- Options name:
- dns_py-dns-crawler
- USES:
- python
- FreshPorts was unable to extract/find any pkg message
- Master Sites:
|
| Commit History - (may be incomplete: for full details, see links to repositories near top of page) |
| Commit | Credits | Log message |
|---|
1.6.5
27 Nov 2024 02:34:19
 |
Po-Chuan Hsieh (sunpoet) |
dns/py-dns-crawler: Update to 1.6.5
Changes: https://gitlab.nic.cz/adam/dns-crawler/-/blob/master/CHANGELOG.md |
1.6.3_2
23 Nov 2024 07:56:10
|
Kai Knoblich (kai) |
*/*: Switch consumers from devel/py-rq to devel/py-rq1
These ports are not compatible with py-rq 2.0.0+ and for some there is
not yet a new version that is ready for py-rq 2.0.0+.
Therefore to avoid regressions, switch the consumers to the newly
created port devel/py-rq1, which uses the 1.16.2 release.
* Bump PORTREVISION due dependency change.
PR: 282649
Approved by: portmgr (blanket) |
1.6.3_1
19 May 2024 17:15:48
|
Po-Chuan Hsieh (sunpoet) |
dns/py-dns-crawler: Allow build with py-setuptools-git-versioning 2.0.0+
- Bump PORTREVISION for package change |
1.6.3
23 Mar 2024 14:49:27
|
Po-Chuan Hsieh (sunpoet) |
dns/py-dns-crawler: Update to 1.6.3
Changes: https://gitlab.nic.cz/adam/dns-crawler/-/blob/master/CHANGELOG.md |
1.6.2
25 Dec 2023 09:00:24
|
Po-Chuan Hsieh (sunpoet) |
dns/py-dns-crawler: Update to 1.6.2
- Convert to USE_PYTHON=pep517
Changes: https://gitlab.nic.cz/adam/dns-crawler/-/blob/master/CHANGELOG.md |
1.5.8
05 Oct 2023 02:00:22
|
Po-Chuan Hsieh (sunpoet) |
dns/py-dns-crawler: Update to 1.5.8
Changes: https://gitlab.nic.cz/adam/dns-crawler/-/blob/master/CHANGELOG.md |
1.4.9_1
29 Sep 2023 14:19:56
|
Po-Chuan Hsieh (sunpoet) |
*: Replace USES=pycryptography* with USE_PYTHON=cryptography{,_build,_test}
- Introduce USE_PYTHON=cryptography{,_build,_test}
- Switch all 96 ports from USES=pycryptography to with
USE_PYTHON=cryptography{,_build,_test}
- Remove Mk/Uses/pycryptography.mk
PR: 273727
Approved by: tcberner (portmgr)
Exp-run by: antoine |
1.4.9_1
31 Jul 2023 16:40:09
|
Po-Chuan Hsieh (sunpoet) |
dns/py-dns-crawler: Sort USES |
1.4.9_1
22 Jul 2023 07:46:38
|
Tobias C. Berner (tcberner) |
framework: convert tree to use USES=pycryptography
A future commit will update to security/py-cryptography will introduce a
rust dependency.
PR: 254853 |
1.4.9_1
27 Jun 2023 19:34:34
|
Rene Ladan (rene) |
all: remove explicit versions in USES=python for "3.x+"
The logic in USES=python will automatically convert this to 3.8+ by
itself.
Adjust two ports that only had Python 3.7 mentioned but build fine
on Python 3.8 too.
finance/quickfix: mark BROKEN with PYTHON
libtool: compile: c++ -DHAVE_CONFIG_H -I. -I../.. -I -I. -I.. -I../.. -I../C++
-DLIBICONV_PLUG -DPYTHON_MAJOR_VERSION=3 -Wno-unused-variable
-Wno-maybe-uninitialized -O2 -pipe -DLIBICONV_PLUG -fstack-protector-strong
-fno-strict-aliasing -DLIBICONV_PLUG -Wall -ansi
-Wno-unused-command-line-argument -Wpointer-arith -Wwrite-strings
-Wno-overloaded-virtual -Wno-deprecated-declarations -Wno-deprecated -std=c++0x
-MT _quickfix_la-QuickfixPython.lo -MD -MP -MF
.deps/_quickfix_la-QuickfixPython.Tpo -c QuickfixPython.cpp -fPIC -DPIC -o
.libs/_quickfix_la-QuickfixPython.o
warning: unknown warning option '-Wno-maybe-uninitialized'; did you mean
'-Wno-uninitialized'? [-Wunknown-warning-option]
QuickfixPython.cpp:175:11: fatal error: 'Python.h' file not found
^~~~~~~~~~
1 warning and 1 error generated.
Reviewed by: portmgr, vishwin, yuri
Differential Revision: <https://reviews.freebsd.org/D40568> |
1.4.9_1
23 Apr 2023 09:09:58
|
Gerald Pfeifer (gerald) |
*: Bump PORTREVISIONs for math/mpc update to 1.3.1 |
1.4.9
30 Jan 2023 13:00:06
|
Po-Chuan Hsieh (sunpoet) |
dns/py-dns-crawler: Update version requirement of RUN_DEPENDS |
1.4.9
11 Jan 2023 15:58:34
|
Dmitry Marakasov (amdmi3) |
*/*: rename CHEESESHOP to PYPI in MASTER_SITES
PR: 267994
Differential revision: D37518
Approved by: bapt |
07 Sep 2022 21:58:51
|
Stefan Eßer (se) |
Remove WWW entries moved into port Makefiles
Commit b7f05445c00f has added WWW entries to port Makefiles based on
WWW: lines in pkg-descr files.
This commit removes the WWW: lines of moved-over URLs from these
pkg-descr files.
Approved by: portmgr (tcberner) |
1.4.9
07 Sep 2022 21:10:59
|
Stefan Eßer (se) |
Add WWW entries to port Makefiles
It has been common practice to have one or more URLs at the end of the
ports' pkg-descr files, one per line and prefixed with "WWW:". These
URLs should point at a project website or other relevant resources.
Access to these URLs required processing of the pkg-descr files, and
they have often become stale over time. If more than one such URL was
present in a pkg-descr file, only the first one was tarnsfered into
the port INDEX, but for many ports only the last line did contain the
port specific URL to further information.
There have been several proposals to make a project URL available as
a macro in the ports' Makefiles, over time.
(Only the first 15 lines of the commit message are shown above  ) |
1.4.9
20 Jul 2022 14:21:47
|
Tobias C. Berner (tcberner) |
dns: remove 'Created by' lines
A big Thank You to the original contributors of these ports:
* Aaron Dalton <aaron@FreeBSD.org>
* Akinori MUSHA aka knu <knu@idaemons.org>
* Alex Samorukov <samm@freebsd.org>
* Alexey Dokuchaev <danfe@FreeBSD.org>
* Allan Jude <allanjude@freebsd.org>
* Amar Takhar <verm@drunkmonk.net>
* Anders Nordby <anders@fix.no>
* Andrew Greenwood <greenwood.andy@gmail.com>
* Anton Berezin <tobez@FreeBSD.org>
* Ashish SHUKLA <ashish@FreeBSD.org>
* Attila Nagy <bra@fsn.hu> (Only the first 15 lines of the commit message are shown above ) |
1.4.9
14 Jun 2021 17:32:32
|
Po-Chuan Hsieh (sunpoet) |
dns/py-dns-crawler: Add py-dns-crawler 1.4.9
Despite the name, the crawler gets info for more services than just DNS:
- DNS:
- all A/AAAA records (for the 2nd level domain and www.subdomain), annotated
with GeoIP
- TXT records (with SPF and DMARC parsed for easier filtering)
- TLSA (for the 2nd level domain and www.subdomain)
- MX
- DNSSEC validation
- nameservers:
- each server IP annotated with GeoIP
- HOSTNAME.BIND, VERSION.BIND, AUTHORS.BIND and fortune (also for all IPs)
- users can add custom additional RRs in the config file
- E-mail (for every server from MX):(Only the first 15 lines of the commit message are shown above ) |
As an Amazon Associate I earn from qualifying purchases.