reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (REEF-2022) AzureBlockBlobFileSystem (REEF) - GetChildren does not work for containers
Date Fri, 08 Jun 2018 16:26:00 GMT

    [ https://issues.apache.org/jira/browse/REEF-2022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16506220#comment-16506220
] 

ASF GitHub Bot commented on REEF-2022:
--------------------------------------

sharathmalladi commented on a change in pull request #1464: [REEF-2022] Fixed AzureBlockBlobFileSystem.GetChildren
api and unit tests
URL: https://github.com/apache/reef/pull/1464#discussion_r190369319
 
 

 ##########
 File path: lang/cs/Org.Apache.REEF.IO.Tests/TestAzureBlockBlobFileSystemE2E.cs
 ##########
 @@ -138,6 +139,54 @@ public void TestDeleteE2E()
             Assert.False(CheckBlobExists(blob));
         }
 
+        [Fact(Skip = SkipMessage)]
+        public void TestGetChildBlobsInContainerE2E()
+        {
+            // setup
+            string[] parentFileNames = new string[] { "sample1", "sample2", "sample3" };
+            string[] folderfileNames = new string[] { "folder1/sample4", "folder1/sample5"
};
+            string[] parentListNames = new string[] { "sample1", "sample2", "sample3", "folder1/"
};
+
+            foreach (string uploadedBlobName in parentFileNames.Concat(folderfileNames))
+            {
+                CloudBlockBlob blob = _container.GetBlockBlobReference(uploadedBlobName);
+                UploadFromString(blob, "hello");
+            }
+
+            // List files in the root level in container
+            ValidateChildenWithBlobs(_container.Uri, parentListNames);
+
+            // List files only in the sub-folder in the container
+            Uri folderUri = _container.GetDirectoryReference("folder1").Uri;
+            ValidateChildenWithBlobs(folderUri, folderfileNames);
+        }
+
+        [Fact(Skip = SkipMessage)]
+        public void TestGetChildContainerInStorageAccountE2E()
+        {
+            // List containers in the storage account
+            Uri rootUri = _fileSystem.CreateUriForPath(string.Empty);
+            ValidateChildrenWithContainers(rootUri, new string[] { _container.Name });
+        }
+
+        public void ValidateChildenWithBlobs(Uri storageBlobUri, IEnumerable<string>
expectedChildBlobNames)
+        {
+            IEnumerable<Uri> blobs = _fileSystem.GetChildren(storageBlobUri).ToList();
+            IEnumerable<string> blobNames = blobs.Select(b => b.LocalPath.Replace("/"
+ _container.Name + "/", string.Empty));
+
+            Assert.True(expectedChildBlobNames.All(blobName => blobNames.Contains(blobName)),
"blobNames has elements that are not in expectedChildBlobNames");
+            Assert.Equal(expectedChildBlobNames.Count(), blobNames.Count());
+        }
+
+        public void ValidateChildrenWithContainers(Uri rootUri, IEnumerable<string>
expectedContainerNames)
 
 Review comment:
   Hmm .. they are different .. in how we extract out the blob names or container names as
the case may be from the output uri. I could try to consolidate it even more but I am afraid
that we lose readability (containers vs blobs). Lets chat offline if you are not convinced
with the reasoning. #Resolved

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> AzureBlockBlobFileSystem (REEF) - GetChildren does not work for containers
> --------------------------------------------------------------------------
>
>                 Key: REEF-2022
>                 URL: https://issues.apache.org/jira/browse/REEF-2022
>             Project: REEF
>          Issue Type: Bug
>          Components: REEF.NET IO
>    Affects Versions: 0.16
>            Reporter: Sharath Malladi
>            Assignee: Sharath Malladi
>            Priority: Major
>
> Repro steps:
> 1. Created a container with multiple files. https://tyclintwscratchstg.blob.core.windows.net/testcontainer/
> 2. Call AzureBlockBlobFileSystem.GetChildren("https://tyclintwscratchstg.blob.core.windows.net/testcontainer/");
>  
> Actual result:
> Throws the exception "The remote server returned an error: (404) Not Found."
>  
> Expected result:
> The files are properly listed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message