libcloud-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [libcloud] rvolykh opened a new issue #1399: Upload large file to Azure Blobs
Date Thu, 02 Jan 2020 16:09:23 GMT
rvolykh opened a new issue #1399: Upload large file to Azure Blobs
URL: https://github.com/apache/libcloud/issues/1399
 
 
   ## Summary
   
   Could not upload a large file (100 GB) to Azure Blob Storage.
   
   ## Detailed Information
   
   Package libcloud: 2.7.0
   File 100.gb was generated with:
   ```
   import os
   f = open('100.gb', "wb")
   f.seek(107374127424-1)
   f.write(b"\0")
   f.close()
   ```
   Code snippet:
   ```
   from io import FileIO
   
   cls = get_driver(Provider.AZURE_BLOBS)
   driver = cls(key='STORAGE_ACCOUNT_NAME', secret='ACCESS_KEY')
   container = driver.get_container(container_name='CONTAINER_NAME')
   
   # This method blocks until all the parts have been uploaded.
   extra = {'content_type': 'application/octet-stream'}
   
   with FileIO('100.gb', 'rb') as iterator:
       obj = driver.upload_object_via_stream(iterator=iterator,
                                             container=container,
                                             object_name='libcloud/100.gb',
                                             extra=extra)
   ```
   Error:
   ```
   Traceback (most recent call last):
     File "/root/rvolykh/venv/lib/python3.5/site-packages/urllib3/connectionpool.py", line
672, in urlopen
       chunked=chunked,
     File "/root/rvolykh/venv/lib/python3.5/site-packages/urllib3/connectionpool.py", line
387, in _make_request
       conn.request(method, url, **httplib_request_kw)
     File "/usr/lib/python3.5/http/client.py", line 1122, in request
       self._send_request(method, url, body, headers)
     File "/usr/lib/python3.5/http/client.py", line 1167, in _send_request
       self.endheaders(body)
     File "/usr/lib/python3.5/http/client.py", line 1118, in endheaders
       self._send_output(message_body)
     File "/usr/lib/python3.5/http/client.py", line 946, in _send_output
       self.send(message_body)
     File "/usr/lib/python3.5/http/client.py", line 915, in send
       self.sock.sendall(datablock)
     File "/usr/lib/python3.5/ssl.py", line 891, in sendall
       v = self.send(data[count:])
     File "/usr/lib/python3.5/ssl.py", line 861, in send
       return self._sslobj.write(data)
     File "/usr/lib/python3.5/ssl.py", line 586, in write
       return self._sslobj.write(data)
   ConnectionResetError: [Errno 104] Connection reset by peer
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message