Issue 4916 - Request compression in CURL WebRequest.

src/DefaultWebRequestCurl.cpp

Index: src/DefaultWebRequestCurl.cpp
===================================================================
--- a/src/DefaultWebRequestCurl.cpp
+++ b/src/DefaultWebRequestCurl.cpp
@@ -133,16 +133,18 @@ AdblockPlus::ServerResponse AdblockPlus:
   if (curl)
   {
     std::stringstream responseText;
     HeaderData headerData;
     curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
     curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
     curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, ReceiveData);
     curl_easy_setopt(curl, CURLOPT_WRITEDATA, &responseText);
+    // Request compressed data. Using any supported aglorithm
+    curl_easy_setopt(curl, CURLOPT_ACCEPT_ENCODING, "");

[Felix Dahlke, 2017/03/02 14:33:38]

Won't it be possible to overwrite this by supplying the `Accept-Encoding` header
in `requestHeaders`? It shouldn't be possible for XHR users to overwrite this,
it's user agent specific after all.

In issue 4916, I suggested to blacklist the headers not allowed in XHR's
`setRequestHeader` function according to the spec, see:
https://codereview.adblockplus.org/29377775/diff/29377796/test/WebRequest.cpp...

I figure the easiest way is to reject those in lib/compat.js. I figure clients
can still overwrite it if they really want to, but it'll be pretty obvious that
this shouldn't be done.

I'm not sure if that should be tackled in the same issue however, feel free to
create a new one for that. I'm not all that worried about it.

[hub, 2017/03/02 15:36:00]

It is my understanding that passing "" will set the only valid values and make
CURL do the proper decoding of the response:
https://github.com/curl/curl/blob/a08db496429e09561c8b8d243e79b21ef04f98f2/do...

An override will cause:
1. no change if the encoding is supported by CURL
2. an error, it the encoding is not supported by CURL

Simply setting the Accept-Encoding header doesn't work because then we need to
handle the decoding out of CURL.

I think blacklisting is a different issue, because with or without this change
it should be done.

[Felix Dahlke, 2017/03/02 15:39:47]

Fair enough.

     curl_easy_setopt(curl, CURLOPT_HEADERFUNCTION, ReceiveHeader);
     curl_easy_setopt(curl, CURLOPT_HEADERDATA, &headerData);
 
     struct curl_slist* headerList = 0;
     for (HeaderList::const_iterator it = requestHeaders.begin();
       it != requestHeaders.end(); ++it)
     {
       headerList = curl_slist_append(headerList, (it->first + ": " + it->second).c_str());