Commit graph

299 commits

Author SHA1 Message Date
Raymond Hill
223815342e
Make dev build use its own assets.json: assets.dev.json
This will make it easier to safely test new versions of
assets.json.
2023-05-04 19:20:18 -04:00
Raymond Hill
e7f6b43659
Update as per filter list changes 2023-04-28 17:02:51 -04:00
Raymond Hill
bc35110180
Pick only the packaged lists 2023-04-27 13:22:15 -04:00
Raymond Hill
ae474aae40
Remove languages with no translations, no activity 2023-04-25 14:21:46 -04:00
Raymond Hill
ba1d686795
Fix github-actions re uBO Lite 2023-04-07 13:34:27 -04:00
Raymond Hill
16b50852c3
Make a firefox package of uBO Lite on release 2023-04-07 13:24:57 -04:00
Raymond Hill
cbfd2ad942
Create a MVP version of uBOLite for Firefox
What does not work at the time of commit:

Cosmetic filtering does not work:

The content scripts responsible for cosmetic filtering fail when
trying to inject the stylesheets through document.adoptedStyleSheets,
with the following error message:

  XrayWrapper denied access to property Symbol.iterator
  (reason: object is not safely Xrayable).
  See https://developer.mozilla.org/en-US/docs/Xray_vision for more
  information. ... css-declarative.js:106:8

A possible solution is to inject those content scripts in the
MAIN world. However Firefox scripting API does not support MAIN
world injection at the moment.

Scriptlet-filtering does not work:

Because scriptlet code needs to be injected in the MAIN world,
and this is currently not supported by Firefox's scripting API,
see https://bugzilla.mozilla.org/show_bug.cgi?id=1736575

There is no count badge on the toolbar icon in Firefox, as it
currently does not support the `DNR.setExtensionActionOptions`
method.

Other than the above issues, it does appear uBO is blocking
properly with no error reported in the dev console.

The adoptedStyleSheets issue though is worrisome, as the
cosmetic filtering content scripts were designed with ISOLATED
world injection in mind. Being forced to inject in MAIN world
(when available) make things a bit more complicated as uBO
has to ensure it's global variables do not leak into the page.
2023-04-07 10:19:43 -04:00
Raymond Hill
64909c671f
Fix crowdin import script 2022-11-29 11:05:21 -05:00
Raymond Hill
65cde3b75a
Remove locales without a translated description 2022-11-18 13:09:27 -05:00
Raymond Hill
f716fe3960
Remove now obsolete shell script re. uAssets 2022-11-14 10:01:56 -05:00
Raymond Hill
17590c5a0c
Use git clone instead of submodule to pull uAssets dependencies
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/2358

We need two different branches of uAssets to properly add the
default filter lists into the packages. I couldn't figure out
how to do this with submodules, using simple `git clone` as a
dependency just work.
2022-11-14 09:50:53 -05:00
Raymond Hill
feaa338678
Code maintenance: replace uDom.js with dom.js
`uDom` is old and crusty and `dom` is meant as replacement. The
goal of `dom` is to be simpler and mainly just convenience
methods for handling the DOM with vanilla JS -- this is not a
framework.

Additionally, removed keyboard shortcuts pane which was useful
only on very old versions of Firefox.
2022-11-12 09:51:22 -05:00
Raymond Hill
985ea24e82
[mv3] Add support for redirect= filters
This adds support for `redirect=` filters. As with `removeparam=`
filters, `redirect=` filters can only be enforced when the
default filtering mode is set to Optimal or Complete, since these
filters require broad host permissions to be enforced by the DNR
engine.

`redirect-rule=` filters are not supported since there is no
corresponding DNR syntax.

Additionally, fixed the dropping of whole network filters even though
those filters are still useful despite not being completely
enforceable -- for example a filter with a single (unsupported) domain
using entity syntax in its `domain=` option should not be wholly
dropped when there are other valid domains in the list.
2022-10-16 12:05:24 -04:00
Raymond Hill
1db3748ab1
[mv3] General code review
Re-arranged resources in a more tidy way. General code review of
various code paths.
2022-10-15 13:05:20 -04:00
Raymond Hill
1a9a19a91f
[mv3] Open options page at first install
To be sure first time users are properly informed about the
possibility to enable broad permissions for better filtering.
2022-09-30 09:18:52 -04:00
Raymond Hill
a71b71e4c8
New cosmetic filter parser using CSSTree library
The new parser no longer uses the browser DOM to validate
that a cosmetic filter is valid or not, this is now done
through a JS library, CSSTree.

This means filter list authors will have to be more careful
to ensure that a cosmetic filter is really valid, as there is
no more guarantee that a cosmetic filter which works for a
given browser/version will still work properly on another
browser, or different version of the same browser.

This change has become necessary because of many reasons,
one of them being the flakiness of the previous parser as
exposed by many issues lately:

- https://github.com/uBlockOrigin/uBlock-issues/issues/2262
- https://github.com/uBlockOrigin/uBlock-issues/issues/2228

The new parser introduces breaking changes, there was no way
to do otherwise. Some current procedural cosmetic filters will
be shown as invalid with this change. This occurs because the
CSSTree library gets confused with some syntax which was
previously allowed by the previous parser because it was more
permissive.

Mainly the issue is with the arguments passed to some procedural
cosmetic filters, and these issues can be solved as follow:

Use quotes around the argument. You can use either single or
double-quotes, whichever is most convenient. If your argument
contains a single quote, use double-quotes, and vice versa.

Additionally, try to escape a quote inside an argument using
backslash. THis may work, but if not, use quotes around the
argument.

When the parser encounter quotes around an argument, it will
discard them before trying to process the argument, same with
escaped quotes inside the argument. Examples:

Breakage:

    ...##^script:has-text(toscr')

Fix:

    ...##^script:has-text(toscr\')

Breakage:

    ...##:xpath(//*[contains(text(),"VPN")]):upward(2)

Fix:

    ...##:xpath('//*[contains(text(),"VPN")]'):upward(2)

There are not many filters which break in the default set of
filter lists, so this should be workable for default lists.

Unfortunately those fixes will break the filter for previous
versions of uBO since these to not deal with quoted argument.
In such case, it may be necessary to keep the previous filter,
which will be discarded as broken on newer version of uBO.

THis was a necessary change as the old parser was becoming
more and more flaky after being constantly patched for new
cases arising, The new parser should be far more robust and
stay robist through expanding procedural cosmetic filter
syntax.

Additionally, in the MV3 version, filters are pre-compiled
using a Nodejs script, i.e. outside the browser, so validating
cosmetic filters using a live DOM no longer made sense.

This new parser will have to be tested throughly before stable
release.
2022-09-23 16:03:13 -04:00
Raymond Hill
70a0de9d00
[mv3] Mind trusted-site directives when registering content scripts 2022-09-20 08:24:01 -04:00
Raymond Hill
adb5d503d9
[mv3] Add GitHub action to build mv3 separately form uBO 2022-09-19 11:08:34 -04:00
Raymond Hill
232c44eeb2
[mv3] Add scriptlet support; improve reliability of cosmetic filtering
First iteration of adding scriptlet support. As with cosmetic
filtering, scriptlet niijection occurs only on sites for which
uBO Lite was granted extended permissions.

At the moment, only three scriptlets are supported:
- abort-current-script
- json-prune
- set-constant

More will be added in the future.
2022-09-16 15:56:35 -04:00
Raymond Hill
99e4fa1d4d
Set zip to quiet mode 2022-09-15 13:37:39 -04:00
Raymond Hill
34aab95107
[mv3] Add support for specific cosmetic filtering
Specific plain CSS cosmetic filters are now supported.

Cosmetic filtering will occur only after the user explicitly
grant uBO extended permissions for a given site, so that it
can inject CSS on the site.

A new button in the popup panel allows a user to grant/revoke
extended permissions to/from uBO Lite for the current site.

More capabilities will be carefully added for when extended
permissions are granted on a site, so specific cosmetic
filtering through plain CSS is the first implemented capability.

Generic and procedural cosmetic filtering is not implemented.

The current implementation for plain CSS cosmetic filters is
through declarative content injection, which does not require
the service worker to be alive, the browser takes care to
inject the cosmetic filters.

However declarative CSS injection does not support user
styles, so the injected cosmetic filters are "weak". I consider
this is a browser issue, since user styles are supported by
Chromium, there is just no way in the API to specify user
styles for the injected content.

Also:
- Fixed dark theme issues
- Added Steven Black's hosts file

Keep in mind all this is very experimental and implementation
details in this release may (will) greatly change in the future.
2022-09-15 13:14:08 -04:00
Raymond Hill
3435e9886e
Import translation work from https://crowdin.com/project/ublock 2022-09-13 17:58:12 -04:00
Raymond Hill
e31637af78
[mv3] Add ability to enable/disable filter lists 2022-09-13 17:44:24 -04:00
Raymond Hill
517f7f5c23
Github action stuff 2022-09-08 12:07:01 -04:00
Raymond Hill
96e1fe49d4
Github action stuff 2022-09-08 11:44:27 -04:00
Raymond Hill
fe52c97392
Github action stuff 2022-09-08 11:38:06 -04:00
Raymond Hill
3d4ac1bc0a
Github action stuff 2022-09-08 11:29:35 -04:00
Raymond Hill
091125ecc5
Github action stuff 2022-09-08 11:21:39 -04:00
Raymond Hill
523bf337ba
Fix github action for mv3 2022-09-08 10:24:17 -04:00
Raymond Hill
f7205fe2e4
Fine tune build steps 2022-09-07 10:23:40 -04:00
Raymond Hill
224410a6f5
Add per-site on/off switch to mv3 experimental version 2022-09-07 10:15:36 -04:00
Raymond Hill
a559f5f271
Add experimental mv3 version
This create a separate Chromium extension, named
"uBO Minus (MV3)".

This experimental mv3 version supports only the blocking of
network requests through the declarativeNetRequest API, so as
to abide by the stated MV3 philosophy of not requiring broad
"read/modify data" permission. Accordingly, the extension
should not trigger the warning at installation time:

    Read and change all your data on all websites

The consequences of being permission-less are the following:

- No cosmetic filtering (##)
- No scriptlet injection (##+js)
- No redirect= filters
- No csp= filters
- No removeparam= filters

At this point there is no popup panel or options pages.

The default filterset correspond to the default filterset of
uBO proper:

Listset for 'default':
  https://ublockorigin.github.io/uAssets/filters/badware.txt
  https://ublockorigin.github.io/uAssets/filters/filters.txt
  https://ublockorigin.github.io/uAssets/filters/filters-2020.txt
  https://ublockorigin.github.io/uAssets/filters/filters-2021.txt
  https://ublockorigin.github.io/uAssets/filters/filters-2022.txt
  https://ublockorigin.github.io/uAssets/filters/privacy.txt
  https://ublockorigin.github.io/uAssets/filters/quick-fixes.txt
  https://ublockorigin.github.io/uAssets/filters/resource-abuse.txt
  https://ublockorigin.github.io/uAssets/filters/unbreak.txt
  https://easylist.to/easylist/easylist.txt
  https://easylist.to/easylist/easyprivacy.txt
  https://malware-filter.gitlab.io/malware-filter/urlhaus-filter-online.txt
  https://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&showintro=1&mimetype=plaintext

The result of the conversion of the filters in all these
filter lists is as follow:

Ruleset size for 'default': 22245
  Good: 21408
  Maybe good (regexes): 127
  redirect-rule= (discarded): 458
  csp= (discarded): 85
  removeparams= (discarded): 22
  Unsupported: 145

The fact that the number of DNR rules are far lower than the
number of network filters reported in uBO comes from the fact
that lists-to-rulesets converter does its best to coallesce
filters into minimal set of rules. Notably, the DNR's
requestDomains condition property allows to create a single
DNR rule out of all pure hostname-based filters.

Regex-based rules are dynamically added at launch time since
they must be validated as valid DNR regexes through
isRegexSupported() API call.

At this point I consider being permission-less the limiting
factor: if broad "read/modify data" permission is to be used,
than there is not much point for an MV3 version over MV2, just
use the MV2 version if you want to benefit all the features
which can't be implemented without broad "read/modify data"
permission.

To locally build the MV3 extension:

    make mv3

Then load the resulting extension directory in the browser
using the "Load unpacked" button.

From now on there will be a uBlock0.mv3.zip package available
in each release.
2022-09-06 13:47:52 -04:00
Raymond Hill
ad5df2b60f
Use python3 instead of python
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/2135
2022-06-10 11:16:49 -04:00
Raymond Hill
986c25fe24
Add Somali language 2021-10-26 09:10:37 -04:00
Raymond Hill
67737b9b27
Fix import script 2021-10-16 09:03:12 -04:00
Raymond Hill
8e296dd671
Fix update-submodules script 2021-09-07 13:50:52 -04:00
paledega
32ce27cf89
Update make-firefox.sh 2021-08-25 08:47:45 +00:00
Manish Jethani
d959c7aabe
Remove globals.js (#3849) 2021-08-23 10:54:16 -04:00
Manish Jethani
b19393d8dc
Add tasks.js module (#3839) 2021-08-19 07:19:20 -04:00
Manish Jethani
f9655b9179
Run npm install in make-npm.sh (#3831) 2021-08-16 12:34:54 -04:00
Manish Jethani
30e46ba64a
Use symlink for node_modules (#3830) 2021-08-16 11:54:03 -04:00
Manish Jethani
36377320ae
Run tests on request data (#3828) 2021-08-16 07:39:09 -04:00
Raymond Hill
087da6407d
Add support for nodejs flavors
The main nodejs flavor is "npm", which is to be used to
lint/test and the publication of an official npm
package -- and by design it has dependencies on mocha,
eslint, etc.

A new flavor "dig" has been created with minimal
dependencies and which purpose is to easily allow to
write specialized code to investigate local code changes
in uBO -- and it's not meant for publication.

Consequently, "make nodejs" has been replaced with
"make npm", and a new "dig" target has been added to the
makefile, to be used for instrumenting local code changes
for investigation purpose.
2021-08-15 10:43:36 -04:00
Manish Jethani
bd8cb8d50f
Use local tmp directory (#3821) 2021-08-14 12:55:20 -04:00
Manish Jethani
bafe824f09
Fix $TMPDIR check in make-nodejs.sh (#3820) 2021-08-14 12:31:17 -04:00
Manish Jethani
6c2856cb7e
Add package-lock.json (#3819) 2021-08-14 11:59:14 -04:00
Raymond Hill
6c4472ce7f
Save/restore node_modules dependencies if present
To avoid hacing to fetch them all over again each time we
launch the test suite.
2021-08-14 11:27:07 -04:00
Manish Jethani
806fe5dbe1
Add Mocha tests (#3815) 2021-08-13 13:08:13 -04:00
Manish Jethani
a56f43f007
Add .npmrc to stop generating package lock file (#3814) 2021-08-13 11:49:52 -04:00
Manish Jethani
daff6b3a34
Add eslint dependency within Node.js package (#3813) 2021-08-13 06:43:20 -04:00