airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] ashb commented on a change in pull request #4091: [AIRFLOW-2524] Update SageMaker hook, operator and sensor for training, tuning and transform
Date Sat, 27 Oct 2018 21:58:41 GMT
ashb commented on a change in pull request #4091: [AIRFLOW-2524] Update SageMaker hook, operator
and sensor for training, tuning and transform
URL: https://github.com/apache/incubator-airflow/pull/4091#discussion_r228727567
 
 

 ##########
 File path: tests/contrib/hooks/test_sagemaker_hook.py
 ##########
 @@ -233,94 +254,82 @@ class TestSageMakerHook(unittest.TestCase):
 
     def setUp(self):
         configuration.load_test_config()
-        db.merge_conn(
-            models.Connection(
-                conn_id='sagemaker_test_conn_id',
-                conn_type='sagemaker',
-                login='access_id',
-                password='access_key',
-                extra=json.dumps(db_config)
-            )
-        )
+
+    @mock.patch.object(SageMakerHook, 'get_iam_conn')
 
 Review comment:
   We have had a problem with overly specific mocks in the past and prefer using moto where
possible for mocking AWS api calls.
   
   For example this would become:
   
   https://github.com/spulec/moto/blob/master/tests/test_iam/test_iam.py#L153-L167
   ```
       @mock_iam
       def test_expand_role(self): 
            conn = boto3.client('iam', region_name='us-east-1')
            conn.create_role(RoleName="test-role", AssumeRolePolicyDocument="some policy")
   
            hook = SageMakerHook(aws_conn_id='sagemaker_test_conn_id') 
            arn = hook.expand_role(some_role)
            self.assertEqual("arn:aws:iam::123456789012:role/test-role")
   ```
   
   This way it still goes through all the Boto3 client-side validation etc, which has caught
some bugs for us in the past.
   
   We should use this similarly for S3 operations, but I don't think moto supports SageMaker
yet, so we'd still have to mock those API calls.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message