Commit graph

1723 commits

Author SHA1 Message Date
Raymond Hill
27e8c8d468
Normalize tabless xhr to image/media in onHeadersReceived()
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/610

The service worker-related issue affects both
Chromium/Firefox: the type of resources fetched
from a service worker are uniformly set to
`xmlhttprequest`, hence losing a key piece of
information for the purpose of accurate content
filtering.
2019-05-31 09:02:07 -04:00
Raymond Hill
8828522fe8
Fix errors with cosmetic filter exception in the logger
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/602
2019-05-28 07:21:16 -04:00
Raymond Hill
85b89fbe63
Fix broken import-from-file in Whitelist pane
Related discussion:
- https://www.reddit.com/r/uBlockOrigin/comments/bt2d1f/
2019-05-26 08:03:44 -04:00
Raymond Hill
a7bfff03d6
Avoid spurious diff at edit time in "My rules"
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/593

The issue was caused by the lack of empty last line, since
the differ taking into account new line characters.
2019-05-25 10:04:31 -04:00
Raymond Hill
80a8750d35
Select existing "Advanced settings" page if any
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/591

Additionally, I added a link to the logger in the
"About" pane in the dashboard in order to be able
to access the logger without having to go through
the popup panel.
2019-05-25 08:31:06 -04:00
Raymond Hill
fb6d69f543
Discard whole filter with bad csp= content
Related discussion:
- https://www.reddit.com/r/uBlockOrigin/comments/bshn7z/

uBO was just removing the bad option, while the whole
filter needs to be discarded.
2019-05-24 15:41:37 -04:00
Raymond Hill
1e9528e2a6
Fix regression affecting *$csp=-like filters
Related discussion:
- https://www.reddit.com/r/uBlockOrigin/comments/bshn7z/filter_question/

Regression introduced in:
- 3f3a1543ea
2019-05-24 12:15:32 -04:00
Raymond Hill
26708b37c1
Integrate bare-bone filter hit stats in the logger
Related issue:
- https://github.com/gorhill/uBlock/issues/983
- https://github.com/gorhill/uBlock/issues/1353

The current implementation reports statistics for all
static filters, and the presentation/featureset is
intentionally minimal: *Do not open issues about this.*
It's still a work in progress and it will be worked on
slowly and thoughtfully over time and as time allows.

Pausing the logger will not pause the collation of
filter hit statistics, thus it is possible to lower
the logger overhead by pausing logger output without
losing filter hit collation.
2019-05-24 11:18:39 -04:00
Raymond Hill
eef76c49ae
Add a link to the remote asset in asset viewer
The link will be present if and only if the content
of the currently viewed asset has been fetched from
a remote location.
2019-05-23 19:29:59 -04:00
Raymond Hill
294ea41fde
Import emergency fix 5a29a21c81 in dev build 2019-05-23 10:22:51 -04:00
Raymond Hill
1f398134f9
Minor code reivew of 4430ec11e2 2019-05-23 08:15:26 -04:00
Raymond Hill
7b8c087fdd
Start using async/await where it makes sense 2019-05-22 19:23:04 -04:00
Raymond Hill
4430ec11e2
Rearrange inner loop of static network filtering engine
The motivations for the re-arrangement:

- Reducing the number of entry points:
  matchStringExactString() has been removed and
  matchString() is simply reused with a modifier parameter
  to enable matching variants.

- Presumption that most matches, if any, occur early with
  the left-most tokens in a URL. This gives a very small
  marginal performance gain as per built-in benchmark.
2019-05-22 17:51:03 -04:00
Raymond Hill
e8c2f7eea3
Fix "Close this window" not working on document-blocked page
Related discussion:
- https://www.reddit.com/r/uBlockOrigin/comments/breeux/
2019-05-21 18:56:59 -04:00
Raymond Hill
32b04fa262
Re-arrange parsing of type options to be order-independent
Related commit:
- 1888033070

This removes the need to place `all` before any negated
type in the list of options.
2019-05-21 14:04:21 -04:00
Raymond Hill
5eff4a027a
Fix https://github.com/gorhill/uBlock/issues/3541 2019-05-20 18:29:28 -04:00
Raymond Hill
1888033070
Add support for all filter option
Related discussion:
- https://www.reddit.com/r/uBlockOrigin/comments/bqnsoa/

The `all` option is equivalent to specifying all
network-based types + `popup`, `document`,
`inline-font`, `inline-script`.

Example from discussion:

    ||bet365.com^$all

Above will block all network requests, block all popups,
prevent inline fonts/scripts from `bet365.com`. EasyList-
compatible syntax does not allow to accomplish that
semantic when using only `||bet365.com^`.

If using specific negated type options along with `all`,
the order in which the options appear is important. In
such case `all` should always be first, followed by
the negated type option(s).
2019-05-20 13:46:36 -04:00
Raymond Hill
72d9758faa
Ensure the "Filter lists" pane is in sync with update status
Related issue:
- https://github.com/gorhill/uBlock/issues/2394

Additionally, I added a new advanced setting to control
how long after launch an auto-update session should be
started -- value is in seconds:

    autoUpdateDelayAfterLaunch 180
2019-05-19 18:31:12 -04:00
Raymond Hill
a0ac1b7ee8
Fix handling of data: for filtering purpose in logger
Related issue:
- https://github.com/gorhill/uBlock/issues/2469
2019-05-19 17:00:49 -04:00
Raymond Hill
f677443878
Warn when navigating away from pane with unsaved changes
Related issue:
- https://github.com/gorhill/uBlock/issues/3271

When navigating away by clicking another pane tab button,
there will be an embedded warning, which can be ignore
in order to proceed to the new pane, or dismissed by
either clicking on the "Stay" button or anywhere else
in the dashboard.

When navigating away by trying to close the tab, there will
be a built-in browser warning asking for confirmation.
2019-05-19 15:35:00 -04:00
Raymond Hill
1caff7429e
Add optional support for generic procedural cosmetic filters
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/131

The new advanced setting and its default value is:

    allowGenericProceduralFilters false

Whenever this setting is toggled, the user is responsible
of forcing a reload of all filter lists so as to allow uBO
to process differently any existing generic procedural
cosmetic filters.
2019-05-18 18:57:32 -04:00
Raymond Hill
ca34bc4f3e
Fix "Revert" button not resetting after saving changes
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/367
2019-05-18 17:48:19 -04:00
Raymond Hill
3cf71835c4
Set default delay for creating selfie to 3 minutes
Related discussion:
- https://www.reddit.com/r/uBlockOrigin/comments/bq49zi/
2019-05-18 14:43:44 -04:00
Raymond Hill
f7bbc80717
Improve "Whitelist pane"; remove now useless built-in switch rule
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/214

Built-in whitelist directives are now rendered differently
than user-defined whitelist directives. Also, removing a
built-in whitelist directive will only cause that directive
to be commented out, so that users do not have to remember
built-in directives should they want to bring them back.

Related issue:
 https://github.com/uBlockOrigin/uBlock-issues/issues/494

The built-in per-site switch rule
`no-scripting: behind-the-scene false` has been removed,
it should not ever be needed since there will always be a
valid root context for main- and sub-frames.
2019-05-18 14:20:05 -04:00
Raymond Hill
de41c1bf53
Fix parsing of recursive !#if-`!#endif directives
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/270
2019-05-18 10:31:04 -04:00
Raymond Hill
62387fb87a
Prevent picker's preview mode from modifying style attribute
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/551

The issue fixes previewing the hiding/unhiding of targeted
elements in the element picker.

However it does not address the case of previewing
`:style(...)` operators -- this would require a much
more complex fix, which I am not sure is worth the
amount of work and increased code complexity.
2019-05-17 19:26:48 -04:00
Raymond Hill
9bfbbfec84
Adjust visual of cosmetic exception filters in logger
The invariant prefixes `##` and `#@#` are now hidden,
allowing to reveal more of the filter itself when the
logger view is narrow.
2019-05-17 11:45:07 -04:00
Raymond Hill
0ca44b847c
Avoid duplicated strings in filterOrigin w/ new approach
The new approach is simpler and should benefit selfie
serialization/unserialization.

This renders stringDeduplicater obsolete -- it has been
removed.
2019-05-17 10:13:58 -04:00
Raymond Hill
1386429382
Fix regression in applying procedural cosmetic filters
Related commit:
- 3573b6b32c
2019-05-16 17:22:20 -04:00
Raymond Hill
3573b6b32c
Add ability to report exception cosmetic filters in the logger
Related issue:
- https://github.com/gorhill/uBlock/issues/127

Additionally, the extended exception filters in the
logger will be rendered with a line-through to more
easily distinguish them from non-exception ones.

Also, opportunistically converted revisited code to
ES6 syntax.
2019-05-16 13:44:49 -04:00
Raymond Hill
fc109c8b7c
Revisit code to benefit from ES6 syntax 2019-05-15 14:49:12 -04:00
Raymond Hill
1fe3b54acc
Fix cosmetic exception filters not applying
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/575

Regression from:
- 93f80eedfa

Specific cosmetic exception filters need to be returned so that
they can be applied to generic cosmetic filters.
2019-05-15 14:43:59 -04:00
Raymond Hill
39e2a03edb
Fix comment 2019-05-14 09:31:51 -04:00
Raymond Hill
a14dcecf8f
Do not assume wildcards fall on label boundaries
Related commit:
- fe0b7a0e0f

Related feedback:
- https://github.com/uBlockOrigin/uBlock-issues/issues/572#issuecomment-492223980
2019-05-14 09:29:45 -04:00
Raymond Hill
93f80eedfa
Refactor runtime storage of specific cosmetic filters
This was a TODO item:
- 07cbae66a4/src/js/cosmetic-filtering.js (L375)

µBlock.staticExtFilteringEngine.HostnameBasedDB has been
re-factored to accomodate the storing of specific cosmetic
filters.

As a result of this refactoring:

- Memory usage has been further decreased
- Performance of selector retrieval marginally
  improved
- New internal representation opens the door
  to use a specialized version of HNTrie, which
  should further improve performance/memory
  usage
2019-05-14 08:52:34 -04:00
Raymond Hill
8a312b9bbb
Support cases with more than one wildcard
Related commit:
- fe0b7a0e0f

Related feedback:
- https://github.com/uBlockOrigin/uBlock-issues/issues/572#issuecomment-492147440
2019-05-14 06:52:13 -04:00
Raymond Hill
fe0b7a0e0f
Relax destination hostname requirements in redirect filters
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/572

Wildcards are now allowed in the hostname part of redirect
filters. There will be an attempt to find the longest
right-hand portion of the hostname with no wildcard. If
no non-empty hostname can be extracted, `*` will be used.
2019-05-13 20:19:10 -04:00
Raymond Hill
1e40f50eb3
Add benchmark method to cosmetic filtering engine
To measure retrieval of site-specific selectors. From
uBO's own dev console:

    µBlock.cosmeticFilteringEngine.benchmark();
2019-05-12 11:41:47 -04:00
Raymond Hill
57890d60ff
Fix incorrect use of this in static method
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/568

Regression from:
- 19ece97b0c
2019-05-11 17:40:55 -04:00
Raymond Hill
8a7e704080
Add support for nth-ancestor operator in HTML filtering
Also opportunitisically converted some code to
ES6's `class`.
2019-05-11 13:21:23 -04:00
Raymond Hill
915c1f1f3c
Report resources blocked by csp= option in logger
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/552
2019-05-11 10:40:34 -04:00
Raymond Hill
12bdd01595
Ensure "Ignore generic cosmetic filters" sticks on Fennec
Related issue:
- https://www.reddit.com/r/uBlockOrigin/comments/blkudl/

The setting was not sticking at first-install time.
2019-05-11 09:04:13 -04:00
Raymond Hill
e59bdb1485
Defuse fixed position on body element in element zapper
The `fixed` style property on the `body` element will be
defused if an overlay element is removed using the element
zapper.

Related:
- https://www.reddit.com/r/uBlockOrigin/comments/bktxtb/scrolling_doesnt_work/emlscyz
2019-05-06 13:32:55 -04:00
Raymond Hill
3692bb4ada
Add HNTrieRef.dump() and STrieRef.dump() as dev tool
To be used at the console, as an investigation tool for
development purpose.

Using it to verify the content of the largest
FilterHostnameDict instance, I spotted an all-uppercase
hostname in the HNTrieRef instance:

µBlock.staticNetFilteringEngine.categories.get(0).get(0x10000000).dict.dump();

Thus the changes to static-net-filtering.js are to fix
the erroneous insertion of filters with uppercase
characters. The single instance found was a hostname entry
in Malware Domain List (TRIANGLESERVICESLTD dot COM).
2019-05-06 11:12:39 -04:00
Raymond Hill
0e4fbefd07
Remove unecessary null placeholders FilterOriginHitSet et al.
The `null` placeholder are not necessary, we can just use
default arguments instead, and add the HNTrieContainer
references if and only if they are instanciated.
2019-05-01 18:54:11 -04:00
Raymond Hill
9e4385243c
Web accessible secrets can be used for at most one second
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/550

Related Chromium issue (I can't access it):
- https://bugs.chromium.org/p/chromium/issues/detail?id=957866

Findings so far: affects browsers based on Chromium 74.
I could not reproduce the issue with either Chromium 73 or
Google Chrome 75.

This commit is a mitigation: to prevent sites from using
uBO's internal WAR secret for tracking purpose. A secret
can be used for at most one second, after which a new secret
is generated.

The original issue related to the implementation of
secret-gated web accessible resources is:
- https://github.com/gorhill/uBlock/issues/2823
2019-04-30 14:36:07 -04:00
Raymond Hill
73e2f25e95
Add new cosmetic procedural operator: :nth-ancestor(n)
The purpose of this new `:nth-ancestor(n)` operator is to
lookup the nth ancestor relative to the currently selected
node.

It is essentially equivalent to `:xpath(..)`, where
ancestor distance is expressed as a number rather than a
sequence of slash-separated `..`.

The rationale to introduce this new procedural selector
is to have a low overhead way to accomplish ancestor
selection.
2019-04-30 09:02:14 -04:00
Raymond Hill
42bf659695
Revert "Order HNTrie nodes alphabetically to allow for early bailout"
This reverts commit f5f9e05071.
2019-04-30 07:00:52 -04:00
Raymond Hill
f5f9e05071
Order HNTrie nodes alphabetically to allow for early bailout
This commit implements the alphabetical ordering of HNTrie
nodes, so as to make it possible to bail out early at
HNTrie.matches() time.

Contrary to what I expected, there is no performance gain
observed to HNTrie.matches() as per benchmarks -- I find
the results perplexing.

Because of this I will revert this commit immediately.
The purpose of this commit is to record the changes so
that I can bring them back to life in the future whenever
I want to investigate further.
2019-04-30 06:47:54 -04:00
Raymond Hill
adabb56dc9
Do not store impossible to match filters in HNTrie
Consider the two following filters:

    example.com
    www.example.com

This commit make it so that if the first filter is
already present in a given HNTrie, the second filter
will not be stored, since HNTrie will _always_
return the first filter as a match whenever the
hostname to match is example.com or any subdomain
of example.com.

The detection of such pointless filters is
virtually free when adding a hostname to an HNTrie
instance (given how data is stored in the trie), so
in practice no overhead is incurred to detect such
pointless filters.

The ability to ignore impossible to match filters
in HNTrie instances will _especially_ benefit those
using large hosts files.

Examples of how this helps using real configurations:

- Default lists:
  444 filters out of 100,382 were ignored as a result
  of this commit.

- Default lists + "Energized Ultimate Protection":
  283,669 filters out of 903,235 were ignored as a
  result of this commit.

Side note: There was no measurable difference between
the two configurations above in the performance of
the matching algorithm as reported by the built-in
benchmark tool.
2019-04-29 13:15:16 -04:00