mxnet-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [incubator-mxnet] kedarbellare commented on a change in pull request #15023: Extend Clojure BERT example
Date Sun, 02 Jun 2019 16:36:17 GMT
kedarbellare commented on a change in pull request #15023: Extend Clojure BERT example
URL: https://github.com/apache/incubator-mxnet/pull/15023#discussion_r289649461
 
 

 ##########
 File path: contrib/clojure-package/examples/bert/fine-tune-bert.ipynb
 ##########
 @@ -489,6 +492,122 @@
     "                                                   :optimizer (optimizer/adam {:learning-rate
5e-6 :episilon 1e-9})\n",
     "                                                   :batch-end-callback (callback/speedometer
batch-size 1)})})\n"
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Explore results from the fine-tuned model\n",
+    "\n",
+    "Now that our model is fitted, we can use it to infer semantic equivalence of arbitrary
sentence pairs. Note that for demonstration purpose we skipped the warmup learning rate schedule
and validation on dev dataset used in the original implementation. This means that our model's
performance will be significantly less than optimal. Please visit [here](https://gluon-nlp.mxnet.io/model_zoo/bert/index.html)
for the complete fine-tuning scripts (using Python and GluonNLP).\n",
+    "\n",
+    "To do inference with our model we need a predictor. It must have a batch size of 1 so
we can feed the model a single sentence pair."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "#'bert.bert-sentence-classification/fine-tuned-predictor"
+      ]
+     },
+     "execution_count": 14,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "(def fine-tuned-prefix \"fine-tune-sentence-bert\")\n",
+    "\n",
+    "(m/save-checkpoint fine-tune-model {:prefix fine-tuned-prefix :epoch 3})\n",
+    "\n",
+    "(def fine-tuned-predictor\n",
+    "    (infer/create-predictor (infer/model-factory fine-tuned-prefix\n",
+    "                                                 [{:name \"data0\" :shape [1 seq-length]
:dtype dtype/FLOAT32 :layout layout/NT}\n",
+    "                                                  {:name \"data1\" :shape [1 seq-length]
:dtype dtype/FLOAT32 :layout layout/NT}\n",
+    "                                                  {:name \"data2\" :shape [1]      
     :dtype dtype/FLOAT32 :layout layout/N}])\n",
+    "                            {:epoch 3}))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Now we can write a function that feeds a sentence pair to the fine-tuned model:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "#'bert.bert-sentence-classification/predict-equivalence"
+      ]
+     },
+     "execution_count": 15,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "(defn predict-equivalence\n",
+    "    [predictor sentence1 sentence2]\n",
+    "    (let [vocab (bert.util/get-vocab)\n",
+    "          processed-test-data (mapv #(pre-processing (:idx->token vocab)\n",
+    "                                                     (:token->idx vocab) %)\n",
+    "                                    [[sentence1 sentence2]])\n",
+    "          prediction (infer/predict-with-ndarray predictor\n",
+    "                                                 [(ndarray/array (slice-inputs-data
processed-test-data 0) [1 seq-length])\n",
+    "                                                  (ndarray/array (slice-inputs-data
processed-test-data 1) [1 seq-length])\n",
+    "                                                  (ndarray/array (slice-inputs-data
processed-test-data 2) [1])])]\n",
+    "      (ndarray/->vec (first prediction))))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 22,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[0.2633881 0.7366119]"
 
 Review comment:
   what's this the output of?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message