The Crawler/Kinetics/download.py script fails when using num_jobs > 1. The videos are downloaded correctly but the clipping with ffmpeg produces a damaged video that doesn't play fully. My solution was to set prefer='threads' in the call to Parallel. The job is still calling subprocess_check_output for downloads and ffmpeg calls so will likely be ran in parallel. I'm posting in case anybody else encounters this frustrating issue. To devs, please reply if you want me to make a pull request with this fix.
The Crawler/Kinetics/download.py script fails when using num_jobs > 1. The videos are downloaded correctly but the clipping with ffmpeg produces a damaged video that doesn't play fully. My solution was to set prefer='threads' in the call to Parallel. The job is still calling subprocess_check_output for downloads and ffmpeg calls so will likely be ran in parallel. I'm posting in case anybody else encounters this frustrating issue. To devs, please reply if you want me to make a pull request with this fix.