Issue 3246 - Strip empty locale files provided by Crowdin

cms/bin/translate.py

Index: cms/bin/translate.py
diff --git a/cms/bin/translate.py b/cms/bin/translate.py
index 5a855be517d056a698b5018f769ac6883708e011..b0ae5ddf3e83eba67843edcdcf53f501cd41d7d0 100644
--- a/cms/bin/translate.py
+++ b/cms/bin/translate.py
@@ -22,7 +22,7 @@ import json
 import logging
 import os
 import posixpath
-import shutil
+import re
 import sys
 import urllib
 import zipfile
@@ -252,6 +252,7 @@ def download_translations(crowdin_api, source_dir, required_locales):
         if f.lower().endswith(".json"):
           os.remove(os.path.join(root, f))
     # Then extract the new ones in place
+    non_empty_file_regexp = re.compile(r"[^\s{}[\]]")
     for member in archive.namelist():
       path, file_name = posixpath.split(member)
       ext = posixpath.splitext(file_name)[1]
@@ -262,9 +263,11 @@ def download_translations(crowdin_api, source_dir, required_locales):
           locale_path, inverted_required_locales[locale],
           *file_path + [file_name]
         )
-        with archive.open(member) as source_file, \
-             open(output_path, "wb") as target_file:
-          shutil.copyfileobj(source_file, target_file)
+        with archive.open(member) as source_file:
+          locale_file_contents = source_file.read()
+          if re.search(non_empty_file_regexp, locale_file_contents):

[Wladimir Palant, 2015/10/29 19:12:35]

I don't think that using regular expressions on JSON data is a good idea. It
makes more sense to check len(json.load(source_file)), this will also ensure
validity of the data. Positive side-effect: we can dump the JSON data formatted
the way we like it (usually means these parameters: ensure_ascii=False,
sort_keys=True, indent=2, separators=(',', ': ')).

Having to handle UTF-8 here makes things a bit more complicated unfortunately,
the simplest solution should be using the codecs module (io module is sadly
incompatible to zipfile in Python 2).

[kzar, 2015/10/30 12:03:03]

Done.

+            with open(output_path, "wb") as target_file:
+              target_file.write(locale_file_contents)
 
 def crowdin_sync(source_dir, crowdin_api_key):
   with FileSource(source_dir) as source: