notbugAs an Amazon Associate I earn from qualifying purchases.
Want a good read? Try FreeBSD Mastery: Jails (IT Mastery Book 15)
Want a good monitor light? See my photosAll times are UTC
Ukraine
2024-11-06 Today brought in a new category: filesystems. Sadly, the website did not process it well. I am working on that. It may take a few days.
2024-11-07 Commit processing is halted. I have a fix for the new category. It is making its way through the system.
UPDATE: the backend is processing the original filesystems commit now. It should take about an hour.
UPDATE: as of 20:30 UTC, the filesystems commit has been processed. Commit processing is now up to date.
Port details
py-w3lib Library of web-related functions
1.21.0 www on this many watch lists=3 search for ports that depend on this port Find issues related to this port Report an issue related to this port View this port on Repology. pkg-fallout 1.21.0Version of this port present on the latest quarterly branch.
Maintainer: rm@FreeBSD.org search for ports maintained by this maintainer
Port Added: 2012-11-13 09:06:47
Last Update: 2023-06-27 19:34:34
Commit Hash: 3d9a815
People watching this port, also watch:: newfile, p5-MIME-Tools, py311-Automat, libssh, py39-beautifulsoup
Also Listed In: python
License: BSD3CLAUSE
WWW:
https://github.com/scrapy/w3lib/
Description:
This is a Python library of web-related functions, such as: - remove comments, or tags from HTML snippets - extract base url from HTML snippets - translate entites on HTML strings - encoding mulitpart/form-data - convert raw HTTP headers to dicts and vice-versa - construct HTTP auth header - converting HTML pages to unicode - RFC-compliant url joining - sanitize urls (like browsers do) - extract arguments from urls
Homepage    cgit ¦ Codeberg ¦ GitHub ¦ GitLab ¦ SVNWeb

Manual pages:
FreshPorts has no man page information for this port.
pkg-plist: as obtained via: make generate-plist
There is no configure plist information for this port.
Dependency lines:
  • ${PYTHON_PKGNAMEPREFIX}w3lib>0:www/py-w3lib@${PY_FLAVOR}
To install the port:
cd /usr/ports/www/py-w3lib/ && make install clean
To add the package, run one of these commands:
  • pkg install www/py-w3lib
  • pkg install py39-w3lib
NOTE: If this package has multiple flavors (see below), then use one of them instead of the name specified above.
NOTE: This is a Python port. Instead of py39-w3lib listed in the above command, you can pick from the names under the Packages section.
PKGNAME: py39-w3lib
Package flavors (<flavor>: <package>)
  • py39: py39-w3lib
distinfo:
TIMESTAMP = 1572735384 SHA256 (w3lib-1.21.0.tar.gz) = 8b1854fef570b5a5fc84d960e025debd110485d73fd283580376104762774315 SIZE (w3lib-1.21.0.tar.gz) = 38153

Packages (timestamps in pop-ups are UTC):
py37-w3lib
ABIaarch64amd64armv6armv7i386powerpcpowerpc64powerpc64le
FreeBSD:13:latest--1.21.0---1.21.0-
FreeBSD:13:quarterly--------
FreeBSD:14:latest--------
FreeBSD:14:quarterly--------
FreeBSD:15:latest--n/a-n/a---
 
py39-w3lib
ABIaarch64amd64armv6armv7i386powerpcpowerpc64powerpc64le
FreeBSD:13:latest--------
FreeBSD:13:quarterly--1.21.0--1.21.01.21.01.21.0
FreeBSD:14:latest--1.21.0--1.21.0-1.21.0
FreeBSD:14:quarterly-----1.21.01.21.01.21.0
FreeBSD:15:latest--n/a-n/a1.21.01.21.01.21.0
 
py311-w3lib
ABIaarch64amd64armv6armv7i386powerpcpowerpc64powerpc64le
FreeBSD:13:latest1.21.01.21.0-1.21.01.21.0---
FreeBSD:13:quarterly1.21.01.21.0-1.21.01.21.0---
FreeBSD:14:latest1.21.01.21.0-1.21.01.21.0---
FreeBSD:14:quarterly1.21.01.21.0-1.21.01.21.0---
FreeBSD:15:latest1.21.01.21.0n/a1.21.0n/a---
 
Dependencies
NOTE: FreshPorts displays only information on required and default dependencies. Optional dependencies are not covered.
Build dependencies:
  1. py39-setuptools>=63.1.0 : devel/py-setuptools@py39
  2. python3.9 : lang/python39
Test dependencies:
  1. python3.9 : lang/python39
Runtime dependencies:
  1. py39-six>0 : devel/py-six@py39
  2. py39-setuptools>=63.1.0 : devel/py-setuptools@py39
  3. python3.9 : lang/python39
This port is required by:
for Run
  1. devel/py-itemloaders
  2. textproc/py-parsel
  3. www/py-scrapy

Configuration Options:
No options to configure
Options name:
www_py-w3lib
USES:
python
FreshPorts was unable to extract/find any pkg message
Master Sites:
Expand this list (2 items)
Collapse this list.
  1. https://files.pythonhosted.org/packages/source/w/w3lib/
  2. https://pypi.org/packages/source/w/w3lib/
Collapse this list.

Number of commits found: 30

Commit History - (may be incomplete: for full details, see links to repositories near top of page)
CommitCreditsLog message
1.21.0
27 Jun 2023 19:34:34
commit hash: 3d9a815d9c5acbb71f4bb07738bdeab4879feacbcommit hash: 3d9a815d9c5acbb71f4bb07738bdeab4879feacbcommit hash: 3d9a815d9c5acbb71f4bb07738bdeab4879feacbcommit hash: 3d9a815d9c5acbb71f4bb07738bdeab4879feacb files touched by this commit
Rene Ladan (rene) search for other commits by this committer
all: remove explicit versions in USES=python for "3.x+"

The logic in USES=python will automatically convert this to 3.8+ by
itself.

Adjust two ports that only had Python 3.7 mentioned but build fine
on Python 3.8 too.

finance/quickfix: mark BROKEN with PYTHON

libtool: compile:  c++ -DHAVE_CONFIG_H -I. -I../.. -I -I. -I.. -I../.. -I../C++
-DLIBICONV_PLUG -DPYTHON_MAJOR_VERSION=3 -Wno-unused-variable
-Wno-maybe-uninitialized -O2 -pipe -DLIBICONV_PLUG -fstack-protector-strong
-fno-strict-aliasing -DLIBICONV_PLUG -Wall -ansi
-Wno-unused-command-line-argument -Wpointer-arith -Wwrite-strings
-Wno-overloaded-virtual -Wno-deprecated-declarations -Wno-deprecated -std=c++0x
-MT _quickfix_la-QuickfixPython.lo -MD -MP -MF
.deps/_quickfix_la-QuickfixPython.Tpo -c QuickfixPython.cpp  -fPIC -DPIC -o
.libs/_quickfix_la-QuickfixPython.o
warning: unknown warning option '-Wno-maybe-uninitialized'; did you mean
'-Wno-uninitialized'? [-Wunknown-warning-option]
QuickfixPython.cpp:175:11: fatal error: 'Python.h' file not found
          ^~~~~~~~~~
1 warning and 1 error generated.

Reviewed by:	portmgr, vishwin, yuri
Differential Revision:	<https://reviews.freebsd.org/D40568>
1.21.0
11 Jan 2023 15:58:34
commit hash: 77d68471aded6907dbbcb5ef50d557e4c69db656commit hash: 77d68471aded6907dbbcb5ef50d557e4c69db656commit hash: 77d68471aded6907dbbcb5ef50d557e4c69db656commit hash: 77d68471aded6907dbbcb5ef50d557e4c69db656 files touched by this commit
Dmitry Marakasov (amdmi3) search for other commits by this committer
*/*: rename CHEESESHOP to PYPI in MASTER_SITES

PR:			267994
Differential revision:	D37518
Approved by:		bapt
07 Sep 2022 21:58:51
commit hash: fb16dfecae4a6efac9f3a78e0b759fb7a3c53de4commit hash: fb16dfecae4a6efac9f3a78e0b759fb7a3c53de4commit hash: fb16dfecae4a6efac9f3a78e0b759fb7a3c53de4commit hash: fb16dfecae4a6efac9f3a78e0b759fb7a3c53de4 files touched by this commit
Stefan Eßer (se) search for other commits by this committer
Remove WWW entries moved into port Makefiles

Commit b7f05445c00f has added WWW entries to port Makefiles based on
WWW: lines in pkg-descr files.

This commit removes the WWW: lines of moved-over URLs from these
pkg-descr files.

Approved by:		portmgr (tcberner)
1.21.0
07 Sep 2022 21:10:59
commit hash: b7f05445c00f2625aa19b4154ebcbce5ed2daa52commit hash: b7f05445c00f2625aa19b4154ebcbce5ed2daa52commit hash: b7f05445c00f2625aa19b4154ebcbce5ed2daa52commit hash: b7f05445c00f2625aa19b4154ebcbce5ed2daa52 files touched by this commit
Stefan Eßer (se) search for other commits by this committer
Add WWW entries to port Makefiles

It has been common practice to have one or more URLs at the end of the
ports' pkg-descr files, one per line and prefixed with "WWW:". These
URLs should point at a project website or other relevant resources.

Access to these URLs required processing of the pkg-descr files, and
they have often become stale over time. If more than one such URL was
present in a pkg-descr file, only the first one was tarnsfered into
the port INDEX, but for many ports only the last line did contain the
port specific URL to further information.

There have been several proposals to make a project URL available as
a macro in the ports' Makefiles, over time.
(Only the first 15 lines of the commit message are shown above View all of this commit message)
1.21.0
07 Apr 2021 08:09:01
commit hash: cf118ccf875508b9a1c570044c93cfcc82bd455ccommit hash: cf118ccf875508b9a1c570044c93cfcc82bd455ccommit hash: cf118ccf875508b9a1c570044c93cfcc82bd455ccommit hash: cf118ccf875508b9a1c570044c93cfcc82bd455c files touched by this commit
Mathieu Arnold (mat) search for other commits by this committer
One more small cleanup, forgotten yesterday.
Reported by:	lwhsu
1.21.0
06 Apr 2021 14:31:07
commit hash: 305f148f482daf30dcf728039d03d019f88344ebcommit hash: 305f148f482daf30dcf728039d03d019f88344ebcommit hash: 305f148f482daf30dcf728039d03d019f88344ebcommit hash: 305f148f482daf30dcf728039d03d019f88344eb files touched by this commit
Mathieu Arnold (mat) search for other commits by this committer
Remove # $FreeBSD$ from Makefiles.
1.21.0
28 Dec 2020 23:02:15
Revision:559531Original commit files touched by this commit
antoine search for other commits by this committer
Drop python 2.7 support from a few ports

With hat:	portmgr
1.21.0
02 Nov 2019 23:21:46
Revision:516376Original commit files touched by this commit
rm search for other commits by this committer
www/py-w3lib: update to 1.21.0
1.20.0
13 Jan 2019 18:18:09
Revision:490161Original commit files touched by this commit
rm search for other commits by this committer
www/py-w3lib: update to 1.20.0
1.19.0
20 Jun 2018 17:05:44
Revision:472884Original commit files touched by this commit
mat search for other commits by this committer
Use PY_FLAVOR for dependencies.

FLAVOR is the current port's flavor, it should not be used outside of
this scope.

Sponsored by:	Absolight
1.19.0
05 Feb 2018 17:08:26
Revision:460984Original commit files touched by this commit
rm search for other commits by this committer
www/py-w3lib: 1.18.0 -> 1.19.0
1.18.0
29 Dec 2017 09:01:54
Revision:457499Original commit files touched by this commit Sanity Test Failure
antoine search for other commits by this committer
Simplify some USES=python

With hat:	portmgr
1.18.0
30 Nov 2017 15:50:34
Revision:455210Original commit files touched by this commit Sanity Test Failure
mat search for other commits by this committer
Convert Python ports to FLAVORS.

  Ports using USE_PYTHON=distutils are now flavored.  They will
  automatically get flavors (py27, py34, py35, py36) depending on what
  versions they support.

  There is also a USE_PYTHON=flavors for ports that do not use distutils
  but need FLAVORS to be set.  A USE_PYTHON=noflavors can be set if
  using distutils but flavors are not wanted.

  A new USE_PYTHON=optsuffix that will add PYTHON_PKGNAMESUFFIX has been
  added to cope with Python ports that did not have the Python
  PKGNAMEPREFIX but are flavored.

  USES=python now also exports a PY_FLAVOR variable that contains the
(Only the first 15 lines of the commit message are shown above View all of this commit message)
1.18.0
22 Sep 2017 20:56:00
Revision:450414Original commit files touched by this commit
rm search for other commits by this committer
www/py-w3lib: update to 1.18.0
1.17.0
09 Feb 2017 20:53:37
Revision:433773Original commit files touched by this commit
rm search for other commits by this committer
www/py-w3lib: update to 1.17.0
1.16.0
09 Dec 2016 22:22:30
Revision:428250Original commit files touched by this commit
rm search for other commits by this committer
www/py-w3lib: update to 1.16.0
1.15.0
02 Dec 2016 09:21:59
Revision:427548Original commit files touched by this commit
mat search for other commits by this committer
http://github.com redirects to https://github.com, spare everyone a redirect.

Sponsored by:	Absolight
1.15.0
21 Aug 2016 16:27:59
Revision:420569Original commit files touched by this commit
rm search for other commits by this committer
www/py-w3lib: update to 1.15.0
1.14.2
15 Apr 2016 17:47:49
Revision:413360Original commit files touched by this commit
rm search for other commits by this committer
www/py-w3lib: update to 1.14.2
1.13.0
01 Apr 2016 14:33:58
Revision:412351Original commit files touched by this commit
mat search for other commits by this committer
Remove ${PORTSDIR}/ from dependencies, categories v, w, x, y, and z.

With hat:	portmgr
Sponsored by:	Absolight
1.13.0
15 Nov 2015 11:44:17
Revision:401695Original commit files touched by this commit
rm search for other commits by this committer
www/py-w3lib: update to 1.13.0
1.12.0
13 Aug 2015 18:44:52
Revision:394182Original commit files touched by this commit
rm search for other commits by this committer
www/py-w3lib: update to 1.12.0
1.11.0
17 Jan 2015 23:56:30
Revision:377275Original commit files touched by this commit
rm search for other commits by this committer
www/py-w3lib: update to 1.11.0
1.7.1
28 Sep 2014 18:21:25
Revision:369482Original commit files touched by this commit
rm search for other commits by this committer
www/py-w3lib: update to 1.7.1; convert to USES
1.6
04 Jun 2014 17:38:30
Revision:356529Original commit files touched by this commit
rm search for other commits by this committer
www/py-w3lib: update to 1.6
1.5_1
15 Feb 2014 14:43:11
Revision:344396Original commit files touched by this commit
rm search for other commits by this committer
- do not use easy_install, convert to auto-generated packing list
- move DIST_SUBDIR upper, where appropriate
- clarify type of BSD license, where this info was available
- bump PORTREVISION
1.5
10 Nov 2013 18:40:49
Revision:333423Original commit files touched by this commit
rm search for other commits by this committer
- update to 1.5
- convert to setuptools (following upstream change)
- update python version (works with 3.x via py-six)
- add staging
- use PYDISTUTILS_AUTOPLIST
1.3
20 Sep 2013 23:36:54
Revision:327776Original commit files touched by this commit
bapt search for other commits by this committer
Add NO_STAGE all over the place in preparation for the staging support (cat:
www)
1.3
04 Jun 2013 15:52:55
Revision:319874Original commit files touched by this commit
rm search for other commits by this committer
- update to 1.3
1.2
13 Nov 2012 09:06:32
Revision:307364Original commit files touched by this commit
rm search for other commits by this committer
This is a Python library of web-related functions, such as:

  - remove comments, or tags from HTML snippets
  - extract base url from HTML snippets
  - translate entites on HTML strings
  - encoding mulitpart/form-data
  - convert raw HTTP headers to dicts and vice-versa
  - construct HTTP auth header
  - converting HTML pages to unicode
  - RFC-compliant url joining
  - sanitize urls (like browsers do)
  - extract arguments from urls

WWW: http://github.com/scrapy/w3lib

Feature safe:	yes

Number of commits found: 30