Commit graph

1884 commits

Author SHA1 Message Date
Raymond Hill
e2fdc1b94b
Support keyboard shortcut to open popup panel
Related discussion:
- https://www.reddit.com/r/uBlockOrigin/comments/dt47s0/
2019-11-09 09:53:31 -05:00
Raymond Hill
7bf5ac0ca2
Code review of commit 4b34d29384
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/778

Related commit:
- 4b34d29384

Mind that `report-uri` or `report-to` could be
used somewhere in a hostname.
2019-11-09 07:32:19 -05:00
Raymond Hill
4b34d29384
Fix regex used to detect forbidden CSP directives
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/778
2019-11-09 06:59:59 -05:00
Raymond Hill
067ef5843c
Ensure that typeof of href attribute is string
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/777
2019-11-08 09:30:59 -05:00
Raymond Hill
ffae19270f
Coallesce mousemove events
Related commit:
- d1995778ff

Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/615
2019-11-06 07:01:13 -05:00
Raymond Hill
d1995778ff
Allow element picker dialog to be moved around
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/615

There is a grab area under the text area, to be used
to move the dialog around.
2019-11-05 12:03:48 -05:00
Raymond Hill
522249bcb5
Fix test against opener tab id in popup filtering code
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/774

Regression from:
- f204d24bf4
2019-11-04 13:34:44 -05:00
Raymond Hill
ec5191004c
Use encodeURIComponent instead of btoa
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/772
2019-11-04 13:12:40 -05:00
Raymond Hill
da253bf80c
Use type="search" for search input fields
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/768

So that the input fields can gain extra built-in
browser widgets useful in search fields.
2019-11-03 13:14:29 -05:00
Raymond Hill
085a8cdbcc
Fine tune cosmetic filtering badge-related code
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/756

As per feedback:
- https://github.com/uBlockOrigin/uBlock-issues/issues/756#issuecomment-549128106
2019-11-03 09:38:36 -05:00
Raymond Hill
571db71318
Fine tune cosmetic filtering badge-related code
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/756

As per various feedbacks:

Added an advanced setting to keep the original behavior,
which can be potentially costly CPU-wise on some sites:

    popupCosmeticFilterBadgeSlow

Default to `false`. Set to `true` to restore original
method of surveying the number of elements hidden as
a result of applying cosmetic filtering.

As suggested by <https://github.com/gwarser>, skip
descendant of nodes which have been found to be a
match in order to potentially increase the number
of nodes which can be surveyed in the alloted time.
2019-11-02 19:03:07 -04:00
Raymond Hill
4afc0764d2
Fix logic mistake in last commit
Related commit:
- 6d935c8925

Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/756
2019-11-02 11:45:18 -04:00
Raymond Hill
6d935c8925
Bail out when counting hidden elements is too expensive
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/756

When trying the number of hidden elements as a result of
cosmetic filtering for popup panel badge purpose, the
code will bail out if this takes too long, and in such
case the badge will be set to `?`, meaning the number
of hidden elements is undetermined.
2019-11-02 09:16:23 -04:00
Raymond Hill
8acc39581c
Fine tune script tags survey scriptlet
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/756

The code used to count expensive-to-survey particular
inline javascript occurrences will be executed if and
only if no other script tags have been found in the
DOM. This is acceptable since the badge count is to be
deemed approximate (as per documentation).
2019-11-01 14:28:15 -04:00
Raymond Hill
987c9c1a21
Fix on-demand no-cosmetic-filtering badge when generichide in use
Related feedback:
- c090d2fde4 (commitcomment-35767596)

Mind that there might not be selectors to match as a
result of `generichide` or `no-cosmetic-filtering`.
2019-11-01 14:01:30 -04:00
Raymond Hill
c090d2fde4
Count hidden elements on-demand only in popup panel
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/756

The badge value for the no-cosmetic-filtering
switch will be evaluated on-demand only, when
the user hover over the switch with the mouse
cursor.

For touch screen displays, a tap on the switch
will cause the badge to be rendered if not
already done, otherwise this will toggle the
switch as usual.
2019-11-01 12:32:34 -04:00
Raymond Hill
7c0294bd5f
Fix spurious leading/trailing wildcards with valid token
Related feedback:
- https://www.reddit.com/r/uBlockOrigin/comments/dpcvfx/

Regression from:
- 7971b22385

Leading/trailing wildcards are useless when a valid
token can be found, and in such case they need to
be removed so as to ensure the proper filter class
is used.
2019-10-31 13:46:06 -04:00
Raymond Hill
d21b9388f3
Fix broken csp= filters when logger is opened
Regression from:
- 7971b22385

Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/761
2019-10-31 12:43:33 -04:00
Raymond Hill
11c56ab540
Minor fine-tuning of URL tokenizer 2019-10-31 11:15:00 -04:00
Raymond Hill
d8975ee580
Improve efficiency of per-site switches badge code
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/756

This is the code used to find out the count values
displayed as badge on the cosmetic filtering and
scripting per-site switches in the popup panel.

The issue is that document.querySelector*() -- used to
find out the number of hidden elements -- is unduly
expensive on large DOM.

The changes in this commit have focused on avoiding the
use of document.querySelector*() as much as possible.

Also, the results are cached for reuse unless DOM
mutations are detected.
2019-10-31 11:07:11 -04:00
Raymond Hill
a69b301d81
Fine-tune new bidi-trie code
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/761
2019-10-29 10:26:34 -04:00
Raymond Hill
5cc797fb47
Add WASM implementation for BidiTrieContainer.matches()
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/761
2019-10-28 13:57:35 -04:00
Raymond Hill
d7b2d31180
Harden compiled/selfie format change detection at launch
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/759

This commit adds code to rely less on the state of the
cache storage to decide whether filter lists should be
re-compiled or whether the selfie is currently valid
at launch time when a change in compiled/selfie format
is detected.
2019-10-27 11:49:05 -04:00
Raymond Hill
b79445320d
Force-unsuspend when starting webRequest listener
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/759

If for some reasons the launch code throws, uBO could
find itself in permanent suspend mode, thus preventing
web pages from loading in Firefox.

Though uBO should not have exceptions thrown during
it's initialization code, this commit will ensure
uBO complete its initialization process should it
ever happen for whatever reason.
2019-10-27 11:41:08 -04:00
Raymond Hill
40de7d6489
Fix WASM memory allocation in bidi-trie
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/764
2019-10-27 08:36:17 -04:00
Raymond Hill
c71624d1da
Fix access to detached buffer when using WASM in bidi-trie
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/764

WebAssembly.Memory.grow() preserves the buffer content when
we grow it, no need to manually copy it. Doing so was
causing an access to a no longer valid ArrayBuffer.
2019-10-26 17:37:47 -04:00
Raymond Hill
0373410635
Fix comments in WASM code 2019-10-26 15:34:40 -04:00
Raymond Hill
6c3296958c
Fix last commit due to bad last second change
Related feedback:
- b0cbc47d9a (commitcomment-35677572)

It seems I completely forgot to test the last
"trivial" change to the WASM code.
2019-10-26 15:25:47 -04:00
Raymond Hill
b0cbc47d9a
Add WASM versions for some bidi-trie methods
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/761

Changes related to above issue made it possible to
create WASM versions of methods used in the bidi-trie.
In this commit, WASM versions for startsWith(), indexOf()
and lastIndexOf() have been implemented.
2019-10-26 13:13:53 -04:00
Raymond Hill
dd2a9faa4c
Use request as its own context when none available
Related feedback:
- https://www.reddit.com/r/uBlockOrigin/comments/dn9zso/

When a network request is tabless and contextless, i.e.
`tabId === -1` and `frameId === -1`, use the URL of
the network request as the context.
2019-10-26 10:18:32 -04:00
Raymond Hill
30393fdcf1
Exclude data type (i.e. csp=) from bidi-trie
We need a `matchAll()` method on the bidi-trie before
we can store filters of type `data` in it.

Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/761

Related commit:
- 7971b22385
2019-10-22 18:14:49 -04:00
Raymond Hill
7971b22385
Expand bidi-trie usage in static network filtering engine
Related issues:
- https://github.com/uBlockOrigin/uBlock-issues/issues/761
- https://github.com/uBlockOrigin/uBlock-issues/issues/528

The previous bidi-trie code could only hold filters which
are plain pattern, i.e. no wildcard characters, and which
had no origin option (`domain=`), right and/or left anchor,
and no `csp=` option.

Example of filters that could be moved into a bidi-trie
data structure:

    &ad_box_
    /w/d/capu.php?z=$script,third-party
    ||liveonlinetv247.com/images/muvixx-150x50-watch-now-in-hd-play-btn.gif

Examples of filters that could NOT be moved to a bidi-trie:

    -adap.$domain=~l-adap.org
    /tsc.php?*&ses=
    ||ibsrv.net/*forumsponsor$domain=[...]
    @@||imgspice.com/jquery.cookie.js|$script
    ||view.atdmt.com^*/iview/$third-party
    ||postimg.cc/image/$csp=[...]

Ideally the filters above should be able to be moved to a
bidi-trie since they are basically plain patterns, or at
least partially moved to a bidi-trie when there is only a
single wildcard (i.e. made of two plain patterns).

Also, there were two distinct bidi-tries in which
plain-pattern filters can be moved to: one for patterns
without hostname anchoring and another one for patterns
with hostname-anchoring. This was required because the
hostname-anchored patterns have an extra condition which
is outside the bidi-trie knowledge.

This commit expands the number of filters which can be
stored in the bidi-trie, and also remove the need to
use two distinct bidi-tries.

- Added ability to associate a pattern with an integer
  in the bidi-trie [1].
    - The bidi-trie match code passes this externally
      provided integer when calling an externally
      provided method used for testing extra conditions
      that may be present for a plain pattern found to
      be matching in the bidi-trie.

- Decomposed existing filters into smaller logical units:
    - FilterPlainLeftAnchored =>
        FilterPatternPlain +
        FilterAnchorLeft
    - FilterPlainRightAnchored =>
        FilterPatternPlain +
        FilterAnchorRight
    - FilterExactMatch =>
        FilterPatternPlain +
        FilterAnchorLeft +
        FilterAnchorRight
    - FilterPlainHnAnchored =>
        FilterPatternPlain +
        FilterAnchorHn
    - FilterWildcard1 =>
        FilterPatternPlain + [
          FilterPatternLeft or
          FilterPatternRight
        ]
    - FilterWildcard1HnAnchored =>
        FilterPatternPlain + [
          FilterPatternLeft or
          FilterPatternRight
        ] +
        FilterAnchorHn
    - FilterGenericHnAnchored =>
        FilterPatternGeneric +
        FilterAnchorHn
    - FilterGenericHnAndRightAnchored =>
        FilterPatternGeneric +
        FilterAnchorRight +
        FilterAnchorHn
    - FilterOriginMixedSet =>
        FilterOriginMissSet +
        FilterOriginHitSet
    - Instances of FilterOrigin[...], FilterDataHolder
      can also be added to a composite filter to
      represent `domain=` and `csp=` options.

- Added a new filter class, FilterComposite, for
  filters which are a combination of two or more
  logical units. A FilterComposite instance is a
  match when *all* filters composing it are a
  match.

Since filters are now encoded into combination of
smaller units, it becomes possible to extract the
FilterPatternPlain component and store it in the
bidi-trie, and use the integer as a handle for the
remaining extra conditions, if any.

Since a single pattern in the bidi-trie may be a
component for different filters, the associated
integer points to a sequence of extra conditions,
and a match occurs as soon as one of the extra
conditions (which may itself be a sequence of
conditions) is fulfilled.

Decomposing filters which are currently single
instance into sequences of smaller logical filters
means increasing the storage and CPU overhead when
evaluating such filters. The CPU overhead is
compensated by the fact that more filters can now
moved into the bidi-trie, where the first match is
efficiently evaluated. The extra conditions have to
be evaluated if and only if there is a match in the
bidi-trie.

The storage overhead is compensated by the
bidi-trie's intrinsic nature of merging similar
patterns.

Furthermore, the storage overhead is reduced by no
longer using JavaScript array to store collection
of filters (which is what FilterComposite is):
the same technique used in [2] is imported to store
sequences of filters.

A sequence of filters is a sequence of integer pairs
where the first integer is an index to an actual
filter instance stored in a global array of filters
(`filterUnits`), while the second integer is an index
to the next pair in the sequence -- which means all
sequences of filters are encoded in one single array
of integers (`filterSequences` => Uint32Array). As
a result, a sequence of filters can be represented by
one single integer -- an index to the first pair --
regardless of the number of filters in the sequence.

This representation is further leveraged to replace
the use of JavaScript array in FilterBucket [3],
which used a JavaScript array to store collection
of filters. Doing so means there is no more need for
FilterPair [4], which purpose was to be a lightweight
representation when there was only two filters in a
collection.

As a result of the above changes, the map of `token`
(integer)  => filter instance (object) used to
associate tokens to filters or collections of filters
is replaced with a more efficient map of `token`
(integer) to filter unit index (integer) to lookup a
filter object from the global `filterUnits` array.

Another consequence of using one single global
array to store all filter instances means we can reuse
existing instances when a logical filter instance is
parameter-less, which is the case for FilterAnchorLeft,
FilterAnchorRight, FilterAnchorHn, the index to these
single instances is reused where needed.

`urlTokenizer` now stores the character codes of the
scanned URL into a bidi-trie buffer, for reuse when
string matching methods are called.

New method: `tokenHistogram()`, used to generate
histograms of occurrences of token extracted from URLs
in built-in benchmark. The top results of the "miss"
histogram are used as "bad tokens", i.e. tokens to
avoid if possible when compiling filter lists.

All plain pattern strings are now stored in the
bidi-trie memory buffer, regardless of whether they
will be used in the trie proper or not.

Three methods have been added to the bidi-trie to test
stored string against the URL which is also stored in
then bidi-trie.

FilterParser is now instanciated on demand and
released when no longer used.

***

[1] 135a45a878/src/js/strie.js (L120)
[2] e94024d350
[3] 135a45a878/src/js/static-net-filtering.js (L1630)
[4] 135a45a878/src/js/static-net-filtering.js (L1566)
2019-10-21 08:15:58 -04:00
Raymond Hill
f2340bef3c
Fix bad returned value in case of empty URL
Though I do no expect the empty URL case
to ever occur, having the tokenizer return
the wrong value if it ever occur could cause
uBO to malfunction.
2019-10-17 17:23:05 -04:00
Raymond Hill
0f19dfde38
Avoid or defer writing back to cache storage at launch
The readTime property is unused for compiled or selfie
assets, and thus writing back to storage to persist
this property is useless for those assets, and an undue
overhead when reading such assets, especially at launch
time.

Assets are always loaded from their compiled or selfie
counterparts at launch.

This commit will prevent those unnecessary storage
write operations.
2019-10-15 11:42:43 -04:00
Raymond Hill
f117c280d0
Fix minor bugs spotted during code review 2019-10-14 09:03:51 -04:00
Raymond Hill
bf697f344a
Log procedural cosmetic exception filters
Related issue:
- https://github.com/gorhill/uBlock/issues/127

Procedural cosmetic exception filters were the
last class of cosmetic exception filters not
being reported in the logger; this commit fixes
this.

Additionally, ensure that a single DOM listener
can't prevent other listeners from being
processed by throwing an exception. Such approach
would have prevented regression leading to
emergency release 1.22.4:
- https://github.com/gorhill/uBlock/releases/tag/1.22.4
2019-09-30 18:21:24 -04:00
Raymond Hill
7ac908a3f8
Fix regression in logger's reverse-lookup of filters
Related commit:
- e1d75ee602
2019-09-30 11:41:43 -04:00
Raymond Hill
e1d75ee602
Prevent reverse-lookup from finding badfilter-ed filters
This could occur when clicking old entries in the logger
which no longer matched the internal state of uBO.

Related feedback:
- https://github.com/MajkiIT/polish-ads-filter/issues/14768#issuecomment-536006312
2019-09-30 08:53:02 -04:00
Raymond Hill
3f074d7908
Use self when testing for vAPI presence 2019-09-30 07:50:35 -04:00
Raymond Hill
a8df19fee4
Rename register-like variables
Use leading `$` instead of trailing `$` to denote
register-like variables, this conveniently allows
to group them together in the debugger.
2019-09-29 13:21:09 -04:00
Raymond Hill
bf99623a54
Add sanity checks
In rare circumstances, it's possible the content
script lose access to the background page, best
to check against this to avoid spurious console
errors.
2019-09-29 12:26:58 -04:00
Raymond Hill
e94024d350
Reduce memory usage in staticExtFilteringEngine.HostnameBasedDB
Using pairs of integers allows the use of a
single integer-only array to store lists of
string indices associated to a specific
hostname.

Memory usage of instances of HostnameBasedDB
as per Chromium's heap snaphshot (bytes):

Before:
    2,459,256 => specific cosmetic filters
      944,152 => scriptlet filtering
          736
    ---------
    3,404,144

After:
    1,947,448 => "
      757,936 => "
          632
    ---------
    2,706,016

Ultimately, using 2 integers for each entry
instead of a single one is still worth it
because this allows the use of one single
integer-only array instead of having to use
an array of arrays for hostnames which have
multiple entries.
2019-09-28 13:36:07 -04:00
Raymond Hill
4bf6503f0a
Store csp= filters into main data structure
This commits make it so that `csp=` filters
are now stored in the same data structures as
all other static network filters rather than
being stored in a separate one.

This internal change is motivated by the wish
to bring session filters to the static network
filtering engine, as has already been done for
the static extended filtering engine in the
following commit:

59c9a34d34
2019-09-28 11:30:26 -04:00
Raymond Hill
efa4ff3bcf
Code review re. dynamically loaded vapi-client-extra.js
Related commit:
- 87d0e456f1

Ensure that the code which depends on extending
`vapi-client.js` is ready to deal with
`vapi-client-extra.js` failing to load.
2019-09-26 15:57:55 -04:00
Raymond Hill
7912e919b6
Code review of session filters commit
Related commits:
- 46d36cb0b0
- 59c9a34d34
2019-09-26 12:40:56 -04:00
Raymond Hill
16261aa6b5
Minor code review for d6ccc9857d
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/736
2019-09-26 09:25:33 -04:00
Raymond Hill
d6ccc9857d
Fix missing newline in merging of sublists
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/736

Regression from:
- e27328f931
2019-09-26 09:13:50 -04:00
Raymond Hill
46d36cb0b0
Code review of session filters commit
Related commit:
- 59c9a34d34
2019-09-25 11:21:34 -04:00
Raymond Hill
28aee88a7b
Fix regression in logger's summary pane
Related feedback:
- 59c9a34d34 (commitcomment-35226346)
2019-09-25 06:14:43 -04:00
Raymond Hill
59c9a34d34
Add ability to quickly create exceptions in logger
This is a feature under development, hidden behind
a new advanced setting, `filterAuthorMode` which
default to `false`.

Ability to point-and-click to create temporary
exception filters for static extended filters (i.e.
cosmetic, scriptlet & html filters) from within
the summary pane in the logger. The button to
toggle on/off temporary exception filter is
labeled `#@#`.

The created exceptions are temporary and will be
lost when restarting uBO, or manually toggling off
the exception filters.

Creating temporary exception filters does not
cause the filter lists to reloaded, and thus there
is no overhead in creating/removing these temporary
exception filters.
2019-09-24 17:05:03 -04:00