uBlock/platform/common/vapi-common.js

193 lines
5.6 KiB
JavaScript
Raw Normal View History

/*******************************************************************************
2016-09-10 14:29:18 +02:00
uBlock Origin - a browser extension to block requests.
Copyright (C) 2014-2015 The uBlock Origin authors
Copyright (C) 2014-present Raymond Hill
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see {http://www.gnu.org/licenses/}.
Home: https://github.com/gorhill/uBlock
*/
// For background page or non-background pages
New cosmetic filter parser using CSSTree library The new parser no longer uses the browser DOM to validate that a cosmetic filter is valid or not, this is now done through a JS library, CSSTree. This means filter list authors will have to be more careful to ensure that a cosmetic filter is really valid, as there is no more guarantee that a cosmetic filter which works for a given browser/version will still work properly on another browser, or different version of the same browser. This change has become necessary because of many reasons, one of them being the flakiness of the previous parser as exposed by many issues lately: - https://github.com/uBlockOrigin/uBlock-issues/issues/2262 - https://github.com/uBlockOrigin/uBlock-issues/issues/2228 The new parser introduces breaking changes, there was no way to do otherwise. Some current procedural cosmetic filters will be shown as invalid with this change. This occurs because the CSSTree library gets confused with some syntax which was previously allowed by the previous parser because it was more permissive. Mainly the issue is with the arguments passed to some procedural cosmetic filters, and these issues can be solved as follow: Use quotes around the argument. You can use either single or double-quotes, whichever is most convenient. If your argument contains a single quote, use double-quotes, and vice versa. Additionally, try to escape a quote inside an argument using backslash. THis may work, but if not, use quotes around the argument. When the parser encounter quotes around an argument, it will discard them before trying to process the argument, same with escaped quotes inside the argument. Examples: Breakage: ...##^script:has-text(toscr') Fix: ...##^script:has-text(toscr\') Breakage: ...##:xpath(//*[contains(text(),"VPN")]):upward(2) Fix: ...##:xpath('//*[contains(text(),"VPN")]'):upward(2) There are not many filters which break in the default set of filter lists, so this should be workable for default lists. Unfortunately those fixes will break the filter for previous versions of uBO since these to not deal with quoted argument. In such case, it may be necessary to keep the previous filter, which will be discarded as broken on newer version of uBO. THis was a necessary change as the old parser was becoming more and more flaky after being constantly patched for new cases arising, The new parser should be far more robust and stay robist through expanding procedural cosmetic filter syntax. Additionally, in the MV3 version, filters are pre-compiled using a Nodejs script, i.e. outside the browser, so validating cosmetic filters using a live DOM no longer made sense. This new parser will have to be tested throughly before stable release.
2022-09-23 22:03:13 +02:00
/* global browser */
2016-09-10 14:29:18 +02:00
'use strict';
2014-11-24 15:48:33 +01:00
2015-02-12 16:50:40 +01:00
/******************************************************************************/
/******************************************************************************/
Refactor selfie generation into a more flexible persistence mechanism The motivation is to address the higher peak memory usage at launch time with 3rd-gen HNTrie when a selfie was present. The selfie generation prior to this change was to collect all filtering data into a single data structure, and then to serialize that whole structure at once into storage (using JSON.stringify). However, HNTrie serialization requires that a large UintArray32 be converted into a plain JS array, which itslef would be indirectly converted into a JSON string. This was the main reason why peak memory usage would be higher at launch from selfie, since the JSON string would need to be wholly unserialized into JS objects, which themselves would need to be converted into more specialized data structures (like that Uint32Array one). The solution to lower peak memory usage at launch is to refactor selfie generation to allow a more piecemeal approach: each filtering component is given the ability to serialize itself rather than to be forced to be embedded in the master selfie. With this approach, the HNTrie buffer can now serialize to its own storage by converting the buffer data directly into a string which can be directly sent to storage. This avoiding expensive intermediate steps such as converting into a JS array and then to a JSON string. As part of the refactoring, there was also opportunistic code upgrade to ES6 and Promise (eventually all of uBO's code will be proper ES6). Additionally, the polyfill to bring getBytesInUse() to Firefox has been revisited to replace the rather expensive previous implementation with an implementation with virtually no overhead.
2019-02-14 19:33:55 +01:00
vAPI.T0 = Date.now();
/******************************************************************************/
vAPI.setTimeout = vAPI.setTimeout || self.setTimeout.bind(self);
2014-11-24 23:20:21 +01:00
/******************************************************************************/
vAPI.webextFlavor = {
major: 0,
Add experimental mv3 version This create a separate Chromium extension, named "uBO Minus (MV3)". This experimental mv3 version supports only the blocking of network requests through the declarativeNetRequest API, so as to abide by the stated MV3 philosophy of not requiring broad "read/modify data" permission. Accordingly, the extension should not trigger the warning at installation time: Read and change all your data on all websites The consequences of being permission-less are the following: - No cosmetic filtering (##) - No scriptlet injection (##+js) - No redirect= filters - No csp= filters - No removeparam= filters At this point there is no popup panel or options pages. The default filterset correspond to the default filterset of uBO proper: Listset for 'default': https://ublockorigin.github.io/uAssets/filters/badware.txt https://ublockorigin.github.io/uAssets/filters/filters.txt https://ublockorigin.github.io/uAssets/filters/filters-2020.txt https://ublockorigin.github.io/uAssets/filters/filters-2021.txt https://ublockorigin.github.io/uAssets/filters/filters-2022.txt https://ublockorigin.github.io/uAssets/filters/privacy.txt https://ublockorigin.github.io/uAssets/filters/quick-fixes.txt https://ublockorigin.github.io/uAssets/filters/resource-abuse.txt https://ublockorigin.github.io/uAssets/filters/unbreak.txt https://easylist.to/easylist/easylist.txt https://easylist.to/easylist/easyprivacy.txt https://malware-filter.gitlab.io/malware-filter/urlhaus-filter-online.txt https://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&showintro=1&mimetype=plaintext The result of the conversion of the filters in all these filter lists is as follow: Ruleset size for 'default': 22245 Good: 21408 Maybe good (regexes): 127 redirect-rule= (discarded): 458 csp= (discarded): 85 removeparams= (discarded): 22 Unsupported: 145 The fact that the number of DNR rules are far lower than the number of network filters reported in uBO comes from the fact that lists-to-rulesets converter does its best to coallesce filters into minimal set of rules. Notably, the DNR's requestDomains condition property allows to create a single DNR rule out of all pure hostname-based filters. Regex-based rules are dynamically added at launch time since they must be validated as valid DNR regexes through isRegexSupported() API call. At this point I consider being permission-less the limiting factor: if broad "read/modify data" permission is to be used, than there is not much point for an MV3 version over MV2, just use the MV2 version if you want to benefit all the features which can't be implemented without broad "read/modify data" permission. To locally build the MV3 extension: make mv3 Then load the resulting extension directory in the browser using the "Load unpacked" button. From now on there will be a uBlock0.mv3.zip package available in each release.
2022-09-06 19:47:52 +02:00
soup: new Set(),
get env() {
return Array.from(this.soup);
}
};
(( ) => {
const ua = navigator.userAgent;
const flavor = vAPI.webextFlavor;
const soup = flavor.soup;
const dispatch = function() {
2018-04-04 18:42:01 +02:00
window.dispatchEvent(new CustomEvent('webextFlavor'));
};
// This is always true.
soup.add('ublock').add('webext');
// Whether this is a dev build.
if ( /^\d+\.\d+\.\d+\D/.test(browser.runtime.getManifest().version) ) {
soup.add('devbuild');
}
2018-04-04 18:42:01 +02:00
if ( /\bMobile\b/.test(ua) ) {
soup.add('mobile');
}
2018-04-04 18:42:01 +02:00
// Asynchronous
if (
2019-09-06 16:38:17 +02:00
browser instanceof Object &&
typeof browser.runtime.getBrowserInfo === 'function'
) {
2019-09-06 16:38:17 +02:00
browser.runtime.getBrowserInfo().then(info => {
flavor.major = parseInt(info.version, 10) || flavor.major;
soup.add(info.vendor.toLowerCase())
.add(info.name.toLowerCase());
2018-04-04 18:42:01 +02:00
dispatch();
});
2019-09-06 16:38:17 +02:00
if ( browser.runtime.getURL('').startsWith('moz-extension://') ) {
soup.add('firefox')
.add('user_stylesheet')
.add('html_filtering');
flavor.major = 91;
}
return;
2018-04-04 18:42:01 +02:00
}
// Synchronous -- order of tests is important
const match = /\bChrom(?:e|ium)\/([\d.]+)/.exec(ua);
if ( match !== null ) {
soup.add('chromium')
.add('user_stylesheet');
flavor.major = parseInt(match[1], 10) || 0;
New cosmetic filter parser using CSSTree library The new parser no longer uses the browser DOM to validate that a cosmetic filter is valid or not, this is now done through a JS library, CSSTree. This means filter list authors will have to be more careful to ensure that a cosmetic filter is really valid, as there is no more guarantee that a cosmetic filter which works for a given browser/version will still work properly on another browser, or different version of the same browser. This change has become necessary because of many reasons, one of them being the flakiness of the previous parser as exposed by many issues lately: - https://github.com/uBlockOrigin/uBlock-issues/issues/2262 - https://github.com/uBlockOrigin/uBlock-issues/issues/2228 The new parser introduces breaking changes, there was no way to do otherwise. Some current procedural cosmetic filters will be shown as invalid with this change. This occurs because the CSSTree library gets confused with some syntax which was previously allowed by the previous parser because it was more permissive. Mainly the issue is with the arguments passed to some procedural cosmetic filters, and these issues can be solved as follow: Use quotes around the argument. You can use either single or double-quotes, whichever is most convenient. If your argument contains a single quote, use double-quotes, and vice versa. Additionally, try to escape a quote inside an argument using backslash. THis may work, but if not, use quotes around the argument. When the parser encounter quotes around an argument, it will discard them before trying to process the argument, same with escaped quotes inside the argument. Examples: Breakage: ...##^script:has-text(toscr') Fix: ...##^script:has-text(toscr\') Breakage: ...##:xpath(//*[contains(text(),"VPN")]):upward(2) Fix: ...##:xpath('//*[contains(text(),"VPN")]'):upward(2) There are not many filters which break in the default set of filter lists, so this should be workable for default lists. Unfortunately those fixes will break the filter for previous versions of uBO since these to not deal with quoted argument. In such case, it may be necessary to keep the previous filter, which will be discarded as broken on newer version of uBO. THis was a necessary change as the old parser was becoming more and more flaky after being constantly patched for new cases arising, The new parser should be far more robust and stay robist through expanding procedural cosmetic filter syntax. Additionally, in the MV3 version, filters are pre-compiled using a Nodejs script, i.e. outside the browser, so validating cosmetic filters using a live DOM no longer made sense. This new parser will have to be tested throughly before stable release.
2022-09-23 22:03:13 +02:00
if ( flavor.major >= 105 ) {
soup.add('native_css_has');
}
2018-04-04 18:42:01 +02:00
}
// Don't starve potential listeners
vAPI.setTimeout(dispatch, 97);
2018-04-04 18:42:01 +02:00
})();
/******************************************************************************/
vAPI.download = function(details) {
2019-09-06 16:38:17 +02:00
if ( !details.url ) { return; }
const a = document.createElement('a');
a.href = details.url;
a.setAttribute('download', details.filename || '');
2017-03-25 14:19:57 +01:00
a.setAttribute('type', 'text/plain');
a.dispatchEvent(new MouseEvent('click'));
};
/******************************************************************************/
vAPI.getURL = browser.runtime.getURL;
2014-12-25 14:53:30 +01:00
/******************************************************************************/
2017-09-26 22:09:35 +02:00
// https://github.com/gorhill/uBlock/issues/3057
// - webNavigation.onCreatedNavigationTarget become broken on Firefox when we
2017-09-26 22:35:06 +02:00
// try to make the popup panel close itself using the original
// `window.open('', '_self').close()`.
2017-09-26 22:09:35 +02:00
2014-12-25 14:53:30 +01:00
vAPI.closePopup = function() {
if ( vAPI.webextFlavor.soup.has('firefox') ) {
2017-09-26 22:35:06 +02:00
window.close();
return;
2017-09-26 22:09:35 +02:00
}
2017-09-26 22:35:06 +02:00
// TODO: try to figure why this was used instead of a plain window.close().
// https://github.com/gorhill/uBlock/commit/b301ac031e0c2e9a99cb6f8953319d44e22f33d2#diff-bc664f26b9c453e0d43a9379e8135c6a
window.open('', '_self').close();
2014-12-25 14:53:30 +01:00
};
/******************************************************************************/
2015-03-14 19:12:05 +01:00
// A localStorage-like object which should be accessible from the
// background page or auxiliary pages.
//
// https://github.com/uBlockOrigin/uBlock-issues/issues/899
// Convert into asynchronous access API.
vAPI.localStorage = {
clear: function() {
vAPI.messaging.send('vapi', {
what: 'localStorage',
fn: 'clear',
});
},
getItemAsync: function(key) {
return vAPI.messaging.send('vapi', {
what: 'localStorage',
fn: 'getItemAsync',
args: [ key ],
});
},
removeItem: function(key) {
return vAPI.messaging.send('vapi', {
what: 'localStorage',
fn: 'removeItem',
args: [ key ],
});
},
setItem: function(key, value = undefined) {
return vAPI.messaging.send('vapi', {
what: 'localStorage',
fn: 'setItem',
args: [ key, value ]
});
},
};
/*******************************************************************************
DO NOT:
- Remove the following code
- Add code beyond the following code
Reason:
- https://github.com/gorhill/uBlock/pull/3721
- uBO never uses the return value from injected content scripts
**/
void 0;