Commit graph

65 commits

Author SHA1 Message Date
Raymond Hill
e27328f931
Work toward modernizing code base: promisification
Swathes of code have been converted to use
Promises/async/await. More left to do.

In the process, a regression affecting the fix to
<https://github.com/uBlockOrigin/uBlock-issues/issues/682>
has been fixed.
2019-09-15 07:58:28 -04:00
Raymond Hill
3eeaba45d9
Cherry-picked ac7825c789 2019-09-07 08:31:32 -04:00
Raymond Hill
e0b8cf24d1
Clear internal cache when loading redirect rules
Related commit:
- 3e5c9e00ab

This fix a regression: newly added redirect rules
could end up not being taken into account unless
uBO was restarted.
2019-08-24 13:48:50 -04:00
Raymond Hill
6c73bd78f4
Fix regression when generating data URI in redirect engine
Related feedback:
- https://www.reddit.com/r/uBlockOrigin/comments/cpxm1v/

Put back erroneously removed code which enable to
generate a `data` URI from already encoded resources.
2019-08-16 13:45:07 -04:00
Raymond Hill
68ae847ba3
Add support for AdGuard's mp4 filter option
Related discussion:
- https://github.com/uBlockOrigin/uBlock-issues/issues/701#issuecomment-520884196

The `mp4` filter option will be converted to `redirect=noopmp4-1s`
internally, and `media` type will be assumed.
2019-08-13 12:30:11 -04:00
Raymond Hill
3e5c9e00ab
Add support for AdGuard's empty option
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/701

The filter option `empty` is converted to `redirect=empty`
by uBO internally; however unlike when the `redirect=`
option is used expressly, the `empty` option does not
require a resource type.

When `empty` is used, only network requests which are meant
to return a text response will be redirected to an empty
response body by uBO -- so `empty` will not work for
resources such as images, media, or other binary resources.
2019-08-13 08:16:21 -04:00
Raymond Hill
7ac7b027f4
Restore ability to redirect xhr to image resources
The ability to redirect xmlhttprequest to binary
resources was lost when redirectable/injectable
resources became immutable in commit
152cea2dfe.

This commit restores the ability to redirect a
xmlhttprequest to a binary resource by making
it possible to derive a data: URI from the
content of binary resources such as images.

Addtionally a redirect to a data: URI can be
forced by prefixing the resource token with `%`.
This is a non-official feature at this point,
i.e. it could be removed at any time.
2019-08-06 10:51:24 -04:00
Raymond Hill
aa73f292ec
Add new static network filter option: redirect-rule=
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/310

The purpose of this new option is to add the ability
to create standalone redirect rule without being forced
to create a block filter (a corresponding block filter
is always created when using the `redirect=`).

Additionally:

The syntax `*$redirect=token,...` is now supported, there
is no need to "trick" the filter parser with
`*/*$redirect=token,...` in order to create redirect rules
which are meant to match all paths.

Filters of the form `|http*://` will be normalized into
two corresponding filters `|https://` and `|http://` so as
to reduce the number of filters in the buckets of
untokenizable filters.
2019-08-03 10:18:47 -04:00
Raymond Hill
47cfba1551
Manually import df97249715 from 1.21.4
Related commit:
- https://github.com/gorhill/uBlock/commit/df972497156e
2019-07-22 06:30:00 -04:00
Raymond Hill
716aae4236
Reverting fix to https://github.com/gorhill/uBlock/issues/3639
Related feedback:
- a54cb2e38f (commitcomment-34387041)

Regression from converting uBO resources into
immutable resources. This affects only Chromium-based
browsers.

This is a quick fix for the dev build, to at least unbreak
the reported case.

I need to research whether the root issue (conflict with HTTPS-E)
is still occurring, and if so the fix is not trivial: I will need
to add code to uBO to fetch and convert binary data into `data:`
URIs.
2019-07-21 20:53:42 -04:00
Raymond Hill
5c449f59f2
[redirect engine] Use shorter name as the official name
- nobab.js => bab-defuser.js
- nofab.js => fuckadblock.js-3.2.0
2019-07-09 11:04:12 -04:00
Raymond Hill
e55cae6232
Fine tune new resources-related code
Make sure the parser is safely compatible with old
resources format -- for those users still using
custom resources (via `userResourcesLocation`).

Prepare code for future fix to
<https://github.com/uBlockOrigin/uBlock-issues/issues/156>:

This commit introduces a new private Map() object,
`uBOSafe`, accessible by all injected scriptlets. This
private safe can be used to store data which can be shared
with different scriptlets. The idea is for scriptlets to
use that safe to graciously deal with the need to install
multiple listeners for the same property.
2019-07-08 08:56:36 -04:00
Raymond Hill
6f5aa947fb
Finalize converting resources.txt into immutable resources
With hindsight, I revised decisions made earlier during
this development cycle:

Un-redirectable scriptlets have been removed from
/web_accessible_resources and instead put in the new
/assets/resources/scriptlets.js, which contains all
scriptlets used for web page injection purpose.

uBO will no longer fetch a remote version of built-in
resources.

Advanced setting `userResourcesLocation` will still be
honoured by uBO, and if set, will be fetched every
time at least one asset is updated.
2019-07-06 12:36:28 -04:00
Raymond Hill
f9e680f111
Convert more resources as immutable
Related commit:
- 152cea2dfe
2019-07-04 14:08:56 -04:00
Raymond Hill
8e245c8919
Convert more resources as immutable
Related commit:
- 152cea2dfe
2019-07-03 19:26:09 -04:00
Raymond Hill
0ba9a35818
Convert more resources as immutable
Related commit:
- 152cea2dfe
2019-07-03 14:33:06 -04:00
Raymond Hill
152cea2dfe
Refactor management of injectable resources
This is a first step, the ultimate goal is to remove
the need for resources.txt, or at least to reduce to
only hotfixes or for trivial resources targeting very
specific websites.

Most resources will become immutable, i.e. they will
be part of uBO's code base. Advantages include easier
code maintenance (jshint, syntax highlight), and to
make scriptlets more easy to code review by external
parties (for example extension store reviewers).

TODO:

- More scriptlets need to be imported before next
  release.
- Need to make legacy versions of uBO use a legacy
  version of resources.txt, as all the now obsolete
  scriptlets will have to be removed once uBO's
  next release become widespread.
- Possibly need to add code to load binary
  resources so that they can be injected as
  data: URI. So far it's unclear whether this is
  really needed. For example, this would be needed
  if a xmlhttprequest is redirected to an image
  resource.
2019-07-03 09:47:56 -04:00
Raymond Hill
a14dcecf8f
Do not assume wildcards fall on label boundaries
Related commit:
- fe0b7a0e0f

Related feedback:
- https://github.com/uBlockOrigin/uBlock-issues/issues/572#issuecomment-492223980
2019-05-14 09:29:45 -04:00
Raymond Hill
8a312b9bbb
Support cases with more than one wildcard
Related commit:
- fe0b7a0e0f

Related feedback:
- https://github.com/uBlockOrigin/uBlock-issues/issues/572#issuecomment-492147440
2019-05-14 06:52:13 -04:00
Raymond Hill
fe0b7a0e0f
Relax destination hostname requirements in redirect filters
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/572

Wildcards are now allowed in the hostname part of redirect
filters. There will be an attempt to find the longest
right-hand portion of the hostname with no wildcard. If
no non-empty hostname can be extracted, `*` will be used.
2019-05-13 20:19:10 -04:00
Raymond Hill
9e4385243c
Web accessible secrets can be used for at most one second
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/550

Related Chromium issue (I can't access it):
- https://bugs.chromium.org/p/chromium/issues/detail?id=957866

Findings so far: affects browsers based on Chromium 74.
I could not reproduce the issue with either Chromium 73 or
Google Chrome 75.

This commit is a mitigation: to prevent sites from using
uBO's internal WAR secret for tracking purpose. A secret
can be used for at most one second, after which a new secret
is generated.

The original issue related to the implementation of
secret-gated web accessible resources is:
- https://github.com/gorhill/uBlock/issues/2823
2019-04-30 14:36:07 -04:00
Raymond Hill
1c26afe874
Remove caching the result of domain extraction from hostname
With the new PSL implementation, benchmarks do not show benefit
from caching the domain extracted from a hostname for later
reuse -- the caching seems to even add an overhead instead with
the new publicSuffixList implementation.
2019-02-20 08:51:14 -05:00
Raymond Hill
ed7e34fb07
Refactor selfie generation into a more flexible persistence mechanism
The motivation is to address the higher peak memory usage at launch
time with 3rd-gen HNTrie when a selfie was present.

The selfie generation prior to this change was to collect all
filtering data into a single data structure, and then to serialize
that whole structure at once into storage (using JSON.stringify).

However, HNTrie serialization requires that a large UintArray32 be
converted into a plain JS array, which itslef would be indirectly
converted into a JSON string. This was the main reason why peak
memory usage would be higher at launch from selfie, since the JSON
string would need to be wholly unserialized into JS objects, which
themselves would need to be converted into more specialized data
structures (like that Uint32Array one).

The solution to lower peak memory usage at launch is to refactor
selfie generation to allow a more piecemeal approach: each filtering
component is given the ability to serialize itself rather than to be
forced to be embedded in the master selfie. With this approach, the
HNTrie buffer can now serialize to its own storage by converting the
buffer data directly into a string which can be directly sent to
storage. This avoiding expensive intermediate steps such as
converting into a JS array and then to a JSON string.

As part of the refactoring, there was also opportunistic code
upgrade to ES6 and Promise (eventually all of uBO's code will be
proper ES6).

Additionally, the polyfill to bring getBytesInUse() to Firefox has
been revisited to replace the rather expensive previous
implementation with an implementation with virtually no overhead.
2019-02-14 13:33:55 -05:00
Raymond Hill
dfcd23197d
Fix parsing of redirect= option as per 67e06f53b4 (commitcomment-27803901) 2018-12-17 07:46:04 -05:00
Raymond Hill
9b27a98f90
Fix https://github.com/gorhill/uBlock/issues/3654
Additionally, there has been refactoring work done regarding
filtering context used throughout uBO, motivated by the fix
here.
2018-12-13 12:30:54 -05:00
Raymond Hill
4504040344
minor code review: do not cache hostname/domain pairs when parsing filters 2018-11-24 12:09:27 -05:00
Raymond Hill
23cd2e1d4a
make "none" a reserved "redirect=" keyword (see deefe87555) 2018-10-31 19:34:54 -03:00
Raymond Hill
58bd6b6974
adjust lz4 ttl as suggested by @gwarser; cache resources selfie as string 2018-08-22 08:13:10 -04:00
Raymond Hill
38aabc937a
reorganize cache storage compression; workaround fix for #2812 2018-08-11 10:39:43 -04:00
Raymond Hill
e21705dea6
fix 2320610019 (commitcomment-29762923) 2018-07-19 09:40:39 -04:00
Raymond Hill
798f8dab9d
reduce baseline memory at selfie-load time 2018-06-01 07:54:31 -04:00
Raymond Hill
10a4a6465e
fix #3639 2018-04-24 05:48:28 -04:00
Raymond Hill
d9161aeffa
fix #3636 2018-03-23 15:03:29 -04:00
Raymond Hill
9ce1f8af08
fix https://www.reddit.com/r/uBlockOrigin/comments/8524cf/my_custom_scriptlets_doesnt_work_what_am_i_doing/ 2018-03-19 10:25:03 -04:00
Raymond Hill
c3f9f6b716
code review: bit stricter code path 2018-02-18 06:08:48 -05:00
Raymond Hill
4ed0d87b7b
code review: prevent redirection to WAR when request is xmlhttprequest 2018-02-16 13:37:20 -05:00
Raymond Hill
287f04b47e
code review for #3474, #2823: simplify management of "web accessible resources" 2018-02-16 09:07:20 -05:00
Raymond Hill
17930cc778
fix #3474, #2823 2018-02-15 17:25:38 -05:00
Raymond Hill
607968de7f
code review: cache most-recently-used pre-filled scriptlets 2017-12-21 17:05:25 -05:00
gorhill
6112a68faf
fix #2984 2017-10-21 13:43:46 -04:00
gorhill
fcf43d972e
tentatively fix issue reported in #2612 re. FFox 24.8.1 2017-05-19 10:12:55 -04:00
Raymond Hill
3b9fd49c50 Assets management refactored (#2314)
* refactoring assets management code

* finalizing refactoring of assets management

* various code review of new assets management code

* fix #2281

* fix #1961

* fix #1293

* fix #1275

* fix update scheduler timing logic

* forward compatibility (to be removed once 1.11+ is widespread)

* more codereview; give admins ability to specify own assets.json

* "assetKey" is more accurate than "path"

* fix group count update when building dom incrementally

* reorganize content (order, added URLs, etc.)

* ability to customize updater through advanced settings

* better spinner icon
2017-01-18 13:17:47 -05:00
gorhill
f6c46ab582 fix #2099 2016-10-26 08:59:15 -04:00
gorhill
0454ad1b1f re. #1070: rewrite redirect engine to use ES6 Sets/Maps 2016-10-10 09:01:05 -04:00
gorhill
bc62af8bf5 support redirection of network requests of type "media" 2016-07-14 13:31:00 -04:00
gorhill
f8e266c2b0 this fixes #1597 2016-04-27 10:08:32 -04:00
gorhill
2d502076d6 code review: disregard case-matching in redirection filters 2016-01-27 12:42:46 -05:00
gorhill
dcb7ba9f7f code review redirection engine: minimize number of entries in inner loop 2016-01-21 18:30:06 -05:00
gorhill
91c2763259 this fixes #1171 2016-01-07 17:30:56 -05:00
gorhill
1365d150db lower parser restrictions for redirect filters 2015-12-23 18:36:57 -05:00