drill-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bridg...@apache.org
Subject drill git commit: move httpd content to storage plugin content
Date Fri, 31 Mar 2017 21:36:04 GMT
Repository: drill
Updated Branches:
  refs/heads/gh-pages 040d4e1aa -> e10bbd81b


move httpd content to storage plugin content


Project: http://git-wip-us.apache.org/repos/asf/drill/repo
Commit: http://git-wip-us.apache.org/repos/asf/drill/commit/e10bbd81
Tree: http://git-wip-us.apache.org/repos/asf/drill/tree/e10bbd81
Diff: http://git-wip-us.apache.org/repos/asf/drill/diff/e10bbd81

Branch: refs/heads/gh-pages
Commit: e10bbd81bc7d99a41e9e064f868c474684a5b1e8
Parents: 040d4e1
Author: Bridget Bevens <bbevens@maprtech.com>
Authored: Fri Mar 31 14:34:13 2017 -0700
Committer: Bridget Bevens <bbevens@maprtech.com>
Committed: Fri Mar 31 14:34:13 2017 -0700

----------------------------------------------------------------------
 ...configuring-drill-to-read-web-server-logs.md | 86 --------------------
 .../plugins/111-httpd-storage-plugin.md         | 86 ++++++++++++++++++++
 2 files changed, 86 insertions(+), 86 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/drill/blob/e10bbd81/_docs/configure-drill/079-configuring-drill-to-read-web-server-logs.md
----------------------------------------------------------------------
diff --git a/_docs/configure-drill/079-configuring-drill-to-read-web-server-logs.md b/_docs/configure-drill/079-configuring-drill-to-read-web-server-logs.md
deleted file mode 100644
index 95a841e..0000000
--- a/_docs/configure-drill/079-configuring-drill-to-read-web-server-logs.md
+++ /dev/null
@@ -1,86 +0,0 @@
----
-title: "Configuring Drill to Read Web Server Logs"
-date: 2017-03-15 19:34:55 UTC
-parent: "Configure Drill"
----
-
-As of version 1.9, Drill can natively ingest and query web server logs. To configure Drill
to read server logs, you must modify the extensions section in the dfs storage plugin configuration,
as shown below:
-
-    "httpd": {
-      "type": "httpd",
-      "logFormat": "%h %t \"%r\" %>s %b \"%{Referer}i\" \"%{user-agent}i\"",
-      "timestampFormat": null
-    }  
-
-{% include startnote.html %}The logFormat section must match the format of the log files,
otherwise Drill cannot correctly parse the logs.{% include endnote.html %}
-
-## HTTPD Format Strings  
-The following table lists the fields that log files can include. The `timestampformat` is
optional, but you can include a format for the time stamp and Drill will parse the times in
the log files into Drill dates.
-
-
-|Format String | Variable Name |
-|--------------|---------------|
-|%a	| connection.client.ip |
-|%{c}a | connection.client.peerip |
-|%A	| connection.server.ip |
-|%B	| response.body.bytes	|
-|%b	| response.body.bytesclf |
-|%{Foobar}C	 | request.cookies.* |
-|%D	| server.process.time	|
-|%{Foobar}e	| server.environment.* |
-|%f	 | server.filename	|
-|%h	| connection.client.host |
-|%H | request.protocol | 
-|%{Foobar}i | request.header.	|
-|%k | connection.keepalivecount	|
-|%l | connection.client.logname	|
-|%L | request.errorlogid	STRING
-|%m | request.method	|
-|%{Foobar}n | server.module_note.*	|
-|%{Foobar}o | response.header.*	|
-|%p | request.server.port.canonical	|
-|%{canonical}p | connection.server.port.canonical	|
-|%{local}p | connection.server.port	|
-|%{remote}p | connection.client.port	|
-|%P | connection.server.child.processid	|
-|%{pid}P | connection.server.child.processid	|
-|%{tid}P | connection.server.child.threadid	|
-|%{hextid}P	| connection.server.child.hexthreadid	|
-|%q	| request.querystring	|
-|%r	| request.firstline	|
-|%R	| request.handler	|
-|%s	| request.status.original	|
-|%>s | request.status.last	|
-|%t | request.receive.time	|
-|%{msec}t | request.receive.time.begin.msec	|
-|%{begin:msec}t | request.receive.time.begin.msec	|
-|%{end:msec}t | request.receive.time.end.msec	|
-|%{usec}t | request.receive.time.begin.usec	|
-|%{begin:usec}t | request.receive.time.begin.usec	|
-|%{end:usec}t | request.receive.time.end.usec	|
-|%{msec_frac}t | request.receive.time.begin.msec_frac	|
-|%{begin:msec_frac}t | request.receive.time.begin.msec_frac	TIME.EPOCH
-|%{end:msec_frac}t | request.receive.time.end.msec_frac	|
-|%{usec_frac}t |	request.receive.time.begin.usec_frac	|
-|%{begin:usec_frac}t |	request.receive.time.begin.usec_frac	|
-|%{end:usec_frac}t | request.receive.time.end.usec_frac	|
-|%T	| response.server.processing.time	|
-|%u	| connection.client.user	|
-|%U	| request.urlpath	|
-|%v	| connection.server.name.canonical	|
-|%V	| connection.server.name	|
-|%X	| response.connection.status	|
-|%I	| request.bytes	|
-|%O	| response.bytes	|
-|%{cookie}i	| request.cookies	|
-|%{set-cookie}o	| response.cookies | 
-|%{user-agent}i	| request.user-agent |
-|%{referer}i | request.referer	|
-
-## Additional Functionality
-In addition to reading raw log files, the following functions are also useful when analyzing
log files:  
-
-* `parse_url(<url>)`:  This function accepts a URL as an argument and returns a map
of the URL's protocol, authority, host, and path.
-* `parse_query( <query_string> )`:  This function accepts a query string and returns
a key/value pairing of the variables submitted in the request.
-
-A function that parses User Agent strings and returns a map of all the pertinent information
is available at: https://github.com/cgivre/drill-useragent-function

http://git-wip-us.apache.org/repos/asf/drill/blob/e10bbd81/_docs/connect-a-data-source/plugins/111-httpd-storage-plugin.md
----------------------------------------------------------------------
diff --git a/_docs/connect-a-data-source/plugins/111-httpd-storage-plugin.md b/_docs/connect-a-data-source/plugins/111-httpd-storage-plugin.md
new file mode 100644
index 0000000..1aa5184
--- /dev/null
+++ b/_docs/connect-a-data-source/plugins/111-httpd-storage-plugin.md
@@ -0,0 +1,86 @@
+---
+title: "HTTPD Storage Plugin"
+date: 2017-03-15 19:34:55 UTC
+parent: "Configure Drill"
+---
+
+As of version 1.9, Drill can natively ingest and query web server logs. To configure Drill
to read server logs, you must modify the extensions section in the dfs storage plugin configuration,
as shown below:
+
+    "httpd": {
+      "type": "httpd",
+      "logFormat": "%h %t \"%r\" %>s %b \"%{Referer}i\" \"%{user-agent}i\"",
+      "timestampFormat": null
+    }  
+
+{% include startnote.html %}The logFormat section must match the format of the log files,
otherwise Drill cannot correctly parse the logs.{% include endnote.html %}
+
+## HTTPD Format Strings  
+The following table lists the fields that log files can include. The `timestampformat` is
optional, but you can include a format for the time stamp and Drill will parse the times in
the log files into Drill dates.
+
+
+|Format String | Variable Name |
+|--------------|---------------|
+|%a	| connection.client.ip |
+|%{c}a | connection.client.peerip |
+|%A	| connection.server.ip |
+|%B	| response.body.bytes	|
+|%b	| response.body.bytesclf |
+|%{Foobar}C	 | request.cookies.* |
+|%D	| server.process.time	|
+|%{Foobar}e	| server.environment.* |
+|%f	 | server.filename	|
+|%h	| connection.client.host |
+|%H | request.protocol | 
+|%{Foobar}i | request.header.	|
+|%k | connection.keepalivecount	|
+|%l | connection.client.logname	|
+|%L | request.errorlogid	STRING
+|%m | request.method	|
+|%{Foobar}n | server.module_note.*	|
+|%{Foobar}o | response.header.*	|
+|%p | request.server.port.canonical	|
+|%{canonical}p | connection.server.port.canonical	|
+|%{local}p | connection.server.port	|
+|%{remote}p | connection.client.port	|
+|%P | connection.server.child.processid	|
+|%{pid}P | connection.server.child.processid	|
+|%{tid}P | connection.server.child.threadid	|
+|%{hextid}P	| connection.server.child.hexthreadid	|
+|%q	| request.querystring	|
+|%r	| request.firstline	|
+|%R	| request.handler	|
+|%s	| request.status.original	|
+|%>s | request.status.last	|
+|%t | request.receive.time	|
+|%{msec}t | request.receive.time.begin.msec	|
+|%{begin:msec}t | request.receive.time.begin.msec	|
+|%{end:msec}t | request.receive.time.end.msec	|
+|%{usec}t | request.receive.time.begin.usec	|
+|%{begin:usec}t | request.receive.time.begin.usec	|
+|%{end:usec}t | request.receive.time.end.usec	|
+|%{msec_frac}t | request.receive.time.begin.msec_frac	|
+|%{begin:msec_frac}t | request.receive.time.begin.msec_frac	TIME.EPOCH
+|%{end:msec_frac}t | request.receive.time.end.msec_frac	|
+|%{usec_frac}t |	request.receive.time.begin.usec_frac	|
+|%{begin:usec_frac}t |	request.receive.time.begin.usec_frac	|
+|%{end:usec_frac}t | request.receive.time.end.usec_frac	|
+|%T	| response.server.processing.time	|
+|%u	| connection.client.user	|
+|%U	| request.urlpath	|
+|%v	| connection.server.name.canonical	|
+|%V	| connection.server.name	|
+|%X	| response.connection.status	|
+|%I	| request.bytes	|
+|%O	| response.bytes	|
+|%{cookie}i	| request.cookies	|
+|%{set-cookie}o	| response.cookies | 
+|%{user-agent}i	| request.user-agent |
+|%{referer}i | request.referer	|
+
+## Additional Functionality
+In addition to reading raw log files, the following functions are also useful when analyzing
log files:  
+
+* `parse_url(<url>)`:  This function accepts a URL as an argument and returns a map
of the URL's protocol, authority, host, and path.
+* `parse_query( <query_string> )`:  This function accepts a query string and returns
a key/value pairing of the variables submitted in the request.
+
+A function that parses User Agent strings and returns a map of all the pertinent information
is available at: https://github.com/cgivre/drill-useragent-function


Mime
View raw message