cloudstack-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From r...@apache.org
Subject [7/8] git commit: updated refs/heads/master to 24d0609
Date Mon, 07 Dec 2015 18:25:37 GMT
CLOUDSTACK-8592: Implement Quota service

Quota service while allowing for scalability will make sure that the cloud is
not exploited by attacks, careless use and program errors. To address this
problem, we propose to employ a quota-enforcement service that allows resource
usage within certain bounds as defined by policies and available quotas for
various entities.  Quota service extends the functionality of usage server to
provide a measurement for the resources used by the accounts and domains using a
common unit referred to as cloud currency in this document. It can be configured
to ensure that your usage won’t exceed the budget allocated to accounts/domain
in cloud currency.  It will let user know how much of the cloud resources he is
using. It will help the cloud admins, if they want, to ensure that a user does
not go beyond his allocated quota. Per usage cycle if a account is found to be
exceeding its quota then it is locked. Locking an account means that it will not
be able to initiat e a new resource allocation request, whether it is more
storage or an additional ip. Needless to say quota service as well as any action
on the account is configurable.

Changes from Github code review:

- Added marvin test for quota plugin API
- removed unused commented code
- debug messages in debug enabled check
- checks for nulls, fixed access to member variables and feature
- changes based on PR comments
- unit tests for UsageTypes
- unit tests for all Cmd classes
- unit tests for all service and manager impls
- try-catch-finally or try-with-resource in dao impls for failsafe db switching
- remove dead code
- add missing quota calculation case (regression fixed)
- replace tabs with spaces in pom.xmls
- quota: though default value for quota_calculated is 0, the usage server
  makes it null while entering usage entries. Flipping the condition so
  as to acocunt for that.
- quotatypes: fix NPE in quota type
- quota framework test fixes
- made statement period configurable
- changed default email templates to reflect the fact that exhausted quota may not result in a locked account
- added quotaUpdateCmd that refreshes quota balances and sends alerts and statements
- report quotaSummary command returns quota balance, quota usage and state for all account
- made UI framework changes to allow for text area input in edit views
- process usage entries that have greater than 0 usage
- orocess quota entries only if tariff is non zero
- if there are credit entries but no balance entry create a dummy balance entry
- remove any credit entries that are before the last balance entry
  when displaying balance statement
- on a rerun the last balance is now getting added

FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Quota+Service+-+FS
PR: https://github.com/apache/cloudstack/pull/768

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>


Project: http://git-wip-us.apache.org/repos/asf/cloudstack/repo
Commit: http://git-wip-us.apache.org/repos/asf/cloudstack/commit/987fcbd4
Tree: http://git-wip-us.apache.org/repos/asf/cloudstack/tree/987fcbd4
Diff: http://git-wip-us.apache.org/repos/asf/cloudstack/diff/987fcbd4

Branch: refs/heads/master
Commit: 987fcbd441fd9f48f72fe19dab94b8bbcc5df2a9
Parents: f30fbe9
Author: Abhinandan Prateek <abhinandan.prateek@shapeblue.com>
Authored: Thu Aug 13 11:42:30 2015 +0530
Committer: Rohit Yadav <rohit.yadav@shapeblue.com>
Committed: Mon Dec 7 23:02:48 2015 +0530

----------------------------------------------------------------------
 .../org/apache/cloudstack/api/ApiConstants.java |   1 +
 api/src/org/apache/cloudstack/api/BaseCmd.java  |   2 +-
 .../command/admin/usage/GetUsageRecordsCmd.java |  24 +
 .../org/apache/cloudstack/usage/UsageTypes.java |   1 +
 .../api/command/test/UsageCmdTest.java          |  19 +
 .../classes/resources/messages.properties       |  41 +
 client/pom.xml                                  |  10 +
 client/tomcatconf/commands.properties.in        |  11 +
 .../com/cloud/upgrade/dao/Upgrade452to460.java  |   2 +-
 .../com/cloud/upgrade/dao/Upgrade461to470.java  |  17 +
 engine/schema/src/com/cloud/usage/UsageVO.java  |  83 +-
 .../src/com/cloud/usage/dao/UsageDao.java       |   4 +
 .../src/com/cloud/usage/dao/UsageDaoImpl.java   |  68 +-
 .../db/src/com/cloud/utils/db/Transaction.java  |  33 +-
 framework/pom.xml                               |   1 +
 framework/quota/pom.xml                         |  73 ++
 .../quota/spring-framework-quota-context.xml    |  34 +
 .../cloudstack/quota/QuotaAlertManager.java     |  26 +
 .../cloudstack/quota/QuotaAlertManagerImpl.java | 418 ++++++++
 .../apache/cloudstack/quota/QuotaManager.java   |  25 +
 .../cloudstack/quota/QuotaManagerImpl.java      | 464 +++++++++
 .../apache/cloudstack/quota/QuotaStatement.java |  26 +
 .../cloudstack/quota/QuotaStatementImpl.java    | 376 +++++++
 .../cloudstack/quota/constant/QuotaConfig.java  |  57 ++
 .../cloudstack/quota/constant/QuotaTypes.java   | 103 ++
 .../cloudstack/quota/dao/QuotaAccountDao.java   |  35 +
 .../quota/dao/QuotaAccountDaoImpl.java          |  74 ++
 .../cloudstack/quota/dao/QuotaBalanceDao.java   |  43 +
 .../quota/dao/QuotaBalanceDaoImpl.java          | 189 ++++
 .../cloudstack/quota/dao/QuotaCreditsDao.java   |  32 +
 .../quota/dao/QuotaCreditsDaoImpl.java          |  78 ++
 .../quota/dao/QuotaEmailTemplatesDao.java       |  27 +
 .../quota/dao/QuotaEmailTemplatesDaoImpl.java   |  74 ++
 .../cloudstack/quota/dao/QuotaTariffDao.java    |  37 +
 .../quota/dao/QuotaTariffDaoImpl.java           | 133 +++
 .../cloudstack/quota/dao/QuotaUsageDao.java     |  35 +
 .../cloudstack/quota/dao/QuotaUsageDaoImpl.java | 116 +++
 .../quota/dao/ServiceOfferingDao.java           |  25 +
 .../quota/dao/ServiceOfferingDaoImpl.java       |  84 ++
 .../cloudstack/quota/dao/UserVmDetailsDao.java  |  27 +
 .../quota/dao/UserVmDetailsDaoImpl.java         |  59 ++
 .../cloudstack/quota/vo/QuotaAccountVO.java     | 149 +++
 .../cloudstack/quota/vo/QuotaBalanceVO.java     | 133 +++
 .../cloudstack/quota/vo/QuotaCreditsVO.java     | 116 +++
 .../quota/vo/QuotaEmailTemplatesVO.java         | 109 +++
 .../cloudstack/quota/vo/QuotaTariffVO.java      | 170 ++++
 .../cloudstack/quota/vo/QuotaUsageVO.java       | 177 ++++
 .../cloudstack/quota/vo/ServiceOfferingVO.java  | 336 +++++++
 .../cloudstack/quota/vo/UserVmDetailVO.java     |  83 ++
 .../quota/QuotaAlertManagerImplTest.java        | 197 ++++
 .../cloudstack/quota/QuotaManagerImplTest.java  | 200 ++++
 .../cloudstack/quota/QuotaStatementTest.java    | 255 +++++
 .../quota/constant/QuotaTypesTest.java          |  48 +
 plugins/database/quota/pom.xml                  |  99 ++
 .../META-INF/cloudstack/quota/module.properties |  18 +
 .../cloudstack/quota/spring-quota-context.xml   |  31 +
 .../cloudstack/api/command/QuotaBalanceCmd.java | 125 +++
 .../cloudstack/api/command/QuotaCreditsCmd.java | 147 +++
 .../api/command/QuotaEmailTemplateListCmd.java  |  60 ++
 .../command/QuotaEmailTemplateUpdateCmd.java    | 122 +++
 .../api/command/QuotaStatementCmd.java          | 141 +++
 .../cloudstack/api/command/QuotaSummaryCmd.java | 110 +++
 .../api/command/QuotaTariffListCmd.java         |  95 ++
 .../api/command/QuotaTariffUpdateCmd.java       | 102 ++
 .../cloudstack/api/command/QuotaUpdateCmd.java  |  72 ++
 .../api/response/QuotaBalanceResponse.java      | 153 +++
 .../api/response/QuotaCreditsResponse.java      |  91 ++
 .../response/QuotaEmailTemplateResponse.java    |  90 ++
 .../api/response/QuotaResponseBuilder.java      |  65 ++
 .../api/response/QuotaResponseBuilderImpl.java  | 516 ++++++++++
 .../response/QuotaStatementItemResponse.java    | 118 +++
 .../api/response/QuotaStatementResponse.java    | 130 +++
 .../api/response/QuotaSummaryResponse.java      | 155 +++
 .../api/response/QuotaTariffResponse.java       | 134 +++
 .../api/response/QuotaTypeResponse.java         |  58 ++
 .../api/response/QuotaUpdateResponse.java       |  38 +
 .../apache/cloudstack/quota/QuotaService.java   |  39 +
 .../cloudstack/quota/QuotaServiceImpl.java      | 299 ++++++
 .../api/command/QuotaBalanceCmdTest.java        |  65 ++
 .../api/command/QuotaCreditsCmdTest.java        |  87 ++
 .../command/QuotaEmailTemplateListCmdTest.java  |  50 +
 .../QuotaEmailTemplateUpdateCmdTest.java        |  68 ++
 .../api/command/QuotaStatementCmdTest.java      |  53 +
 .../api/command/QuotaTariffListCmdTest.java     |  62 ++
 .../api/command/QuotaTariffUpdateCmdTest.java   |  67 ++
 .../response/QuotaResponseBuilderImplTest.java  | 228 +++++
 .../cloudstack/quota/QuotaServiceImplTest.java  | 183 ++++
 plugins/hypervisors/ovm3/pom.xml                |   2 +-
 plugins/network-elements/nuage-vsp/pom.xml      |   2 +-
 plugins/pom.xml                                 |   1 +
 pom.xml                                         |  16 +-
 .../cloud/api/dispatch/ParamProcessWorker.java  |   8 +
 .../src/com/cloud/user/AccountManagerImpl.java  |   3 +-
 .../api/dispatch/ParamProcessWorkerTest.java    |   5 +
 setup/db/db/schema-421to430-cleanup.sql         |   1 -
 setup/db/db/schema-461to470.sql                 | 110 +++
 test/integration/smoke/test_quota.py            | 204 ++++
 tools/apidoc/gen_toc.py                         |   2 +
 ui/dictionary.jsp                               |  44 +
 ui/plugins/plugins.js                           |   3 +-
 ui/plugins/quota/config.js                      |  25 +
 ui/plugins/quota/icon.png                       | Bin 0 -> 3289 bytes
 ui/plugins/quota/quota.css                      |  68 ++
 ui/plugins/quota/quota.js                       | 971 +++++++++++++++++++
 ui/scripts/ui/dialog.js                         |   4 +-
 ui/scripts/ui/widgets/detailView.js             |  24 +-
 ui/scripts/ui/widgets/listView.js               |  20 +-
 usage/pom.xml                                   |   9 +-
 usage/resources/usageApplicationContext.xml     |  47 +-
 usage/src/com/cloud/usage/UsageManagerImpl.java |  31 +-
 110 files changed, 10350 insertions(+), 81 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/cloudstack/blob/987fcbd4/api/src/org/apache/cloudstack/api/ApiConstants.java
----------------------------------------------------------------------
diff --git a/api/src/org/apache/cloudstack/api/ApiConstants.java b/api/src/org/apache/cloudstack/api/ApiConstants.java
index 5365e14..742d2f4 100644
--- a/api/src/org/apache/cloudstack/api/ApiConstants.java
+++ b/api/src/org/apache/cloudstack/api/ApiConstants.java
@@ -273,6 +273,7 @@ public class ApiConstants {
     public static final String VIRTUAL_MACHINE_ID_IP = "vmidipmap";
     public static final String VIRTUAL_MACHINE_COUNT = "virtualmachinecount";
     public static final String USAGE_ID = "usageid";
+    public static final String USAGE_TYPE = "usagetype";
 
     public static final String VLAN = "vlan";
     public static final String VLAN_RANGE = "vlanrange";

http://git-wip-us.apache.org/repos/asf/cloudstack/blob/987fcbd4/api/src/org/apache/cloudstack/api/BaseCmd.java
----------------------------------------------------------------------
diff --git a/api/src/org/apache/cloudstack/api/BaseCmd.java b/api/src/org/apache/cloudstack/api/BaseCmd.java
index ad3a88c..360b277 100644
--- a/api/src/org/apache/cloudstack/api/BaseCmd.java
+++ b/api/src/org/apache/cloudstack/api/BaseCmd.java
@@ -95,7 +95,7 @@ public abstract class BaseCmd {
         GET, POST, PUT, DELETE
     }
     public static enum CommandType {
-        BOOLEAN, DATE, FLOAT, INTEGER, SHORT, LIST, LONG, OBJECT, MAP, STRING, TZDATE, UUID
+        BOOLEAN, DATE, FLOAT, DOUBLE, INTEGER, SHORT, LIST, LONG, OBJECT, MAP, STRING, TZDATE, UUID
     }
 
     private Object _responseObject;

http://git-wip-us.apache.org/repos/asf/cloudstack/blob/987fcbd4/api/src/org/apache/cloudstack/api/command/admin/usage/GetUsageRecordsCmd.java
----------------------------------------------------------------------
diff --git a/api/src/org/apache/cloudstack/api/command/admin/usage/GetUsageRecordsCmd.java b/api/src/org/apache/cloudstack/api/command/admin/usage/GetUsageRecordsCmd.java
index fd8173d..4cceb3b 100644
--- a/api/src/org/apache/cloudstack/api/command/admin/usage/GetUsageRecordsCmd.java
+++ b/api/src/org/apache/cloudstack/api/command/admin/usage/GetUsageRecordsCmd.java
@@ -111,6 +111,30 @@ public class GetUsageRecordsCmd extends BaseListCmd {
     public String getUsageId() {
         return usageId;
     }
+    public void setAccountName(String accountName) {
+        this.accountName = accountName;
+    }
+
+    public void setDomainId(Long domainId) {
+        this.domainId = domainId;
+    }
+
+    public void setEndDate(Date endDate) {
+        this.endDate = endDate == null ? null : new Date(endDate.getTime());
+    }
+
+    public void setStartDate(Date startDate) {
+        this.startDate = startDate == null ? null : new Date(startDate.getTime());
+    }
+
+    public void setAccountId(Long accountId) {
+        this.accountId = accountId;
+    }
+
+    public void setUsageId(String usageId) {
+        this.usageId = usageId;
+    }
+
 
     /////////////////////////////////////////////////////
     /////////////// API Implementation///////////////////

http://git-wip-us.apache.org/repos/asf/cloudstack/blob/987fcbd4/api/src/org/apache/cloudstack/usage/UsageTypes.java
----------------------------------------------------------------------
diff --git a/api/src/org/apache/cloudstack/usage/UsageTypes.java b/api/src/org/apache/cloudstack/usage/UsageTypes.java
index 3edfd0b..d9cfc13 100644
--- a/api/src/org/apache/cloudstack/usage/UsageTypes.java
+++ b/api/src/org/apache/cloudstack/usage/UsageTypes.java
@@ -22,6 +22,7 @@ import java.util.List;
 import org.apache.cloudstack.api.response.UsageTypeResponse;
 
 public class UsageTypes {
+    /* Any changes here should also reflect in cloud_usage.quota_mapping table */
     public static final int RUNNING_VM = 1;
     public static final int ALLOCATED_VM = 2; // used for tracking how long storage has been allocated for a VM
     public static final int IP_ADDRESS = 3;

http://git-wip-us.apache.org/repos/asf/cloudstack/blob/987fcbd4/api/test/org/apache/cloudstack/api/command/test/UsageCmdTest.java
----------------------------------------------------------------------
diff --git a/api/test/org/apache/cloudstack/api/command/test/UsageCmdTest.java b/api/test/org/apache/cloudstack/api/command/test/UsageCmdTest.java
index cadf275..e5f3e27 100644
--- a/api/test/org/apache/cloudstack/api/command/test/UsageCmdTest.java
+++ b/api/test/org/apache/cloudstack/api/command/test/UsageCmdTest.java
@@ -17,6 +17,7 @@
 package org.apache.cloudstack.api.command.test;
 
 import java.util.ArrayList;
+import java.util.Date;
 import java.util.List;
 
 import junit.framework.TestCase;
@@ -70,4 +71,22 @@ public class UsageCmdTest extends TestCase {
 
     }
 
+    @Test
+    public void testCrud() {
+        getUsageRecordsCmd.setDomainId(1L);
+        assertTrue(getUsageRecordsCmd.getDomainId().equals(1L));
+
+        getUsageRecordsCmd.setAccountName("someAccount");
+        assertTrue(getUsageRecordsCmd.getAccountName().equals("someAccount"));
+
+        Date d = new Date();
+        getUsageRecordsCmd.setStartDate(d);
+        getUsageRecordsCmd.setEndDate(d);
+        assertTrue(getUsageRecordsCmd.getStartDate().equals(d));
+        assertTrue(getUsageRecordsCmd.getEndDate().equals(d));
+
+        getUsageRecordsCmd.setUsageId("someId");
+        assertTrue(getUsageRecordsCmd.getUsageId().equals("someId"));
+    }
+
 }

http://git-wip-us.apache.org/repos/asf/cloudstack/blob/987fcbd4/client/WEB-INF/classes/resources/messages.properties
----------------------------------------------------------------------
diff --git a/client/WEB-INF/classes/resources/messages.properties b/client/WEB-INF/classes/resources/messages.properties
index 9c45015..e7beaa9 100644
--- a/client/WEB-INF/classes/resources/messages.properties
+++ b/client/WEB-INF/classes/resources/messages.properties
@@ -1423,6 +1423,47 @@ label.show.advanced.settings=Show advanced settings
 label.delete.OpenDaylight.device=Delete OpenDaylight Controller
 label.polling.interval.sec=Polling Interval (in sec)
 label.quiet.time.sec=Quiet Time (in sec)
+label.usage.type=Usage Type
+label.usage.unit=Unit
+label.quota.value=Quota Value
+label.quota.description=Quota Description
+label.quota.configuration=Quota Configuration
+label.quota.configure=Configure Quota
+label.quota.remove=Remove Quota
+label.quota.totalusage=Total Usage
+label.quota.balance=Balance
+label.quota.minbalance=Min Balance
+label.quota.enforcequota=Enforce Quota
+label.quota.summary=Summary
+label.quota.fullsummary=All Accounts
+label.quota.tariff=Tariff
+label.quota.state=State
+label.quota.startdate=Start Date
+label.quota.enddate=End Date
+label.quota.total=Total
+label.quota.startquota=Start Quota
+label.quota.endquota=End Quota
+label.quota.type.name=Usage Type
+label.quota.type.unit=Usage Unit
+label.quota.usage=Quota Consumption
+label.quota.add.credits=Add Credits
+label.quota.email.template=Email Template
+label.quota.statement=Statement
+label.quota.statement.balance=Quota Balance
+label.quota.statement.quota=Quota Usage
+label.quota.statement.tariff=Quota Tariff
+label.quota.tariff.value=Tariff Value
+label.quota.tariff.edit=Edit Tariff
+label.quota.tariff.effectivedate=Effective Date
+label.quota.date=Date
+label.quota.dates=Update Dates
+label.quota.credit=Credit
+label.quota.credits=Credits
+label.quota.value=Quota Value
+label.quota.statement.bydates=Statement
+label.quota.email.subject=Subject
+label.quota.email.body=Body
+label.quota.email.lastupdated=Last Update
 label.destroy.vm.graceperiod=Destroy VM Grace Period
 label.SNMP.community=SNMP Community
 label.SNMP.port=SNMP Port

http://git-wip-us.apache.org/repos/asf/cloudstack/blob/987fcbd4/client/pom.xml
----------------------------------------------------------------------
diff --git a/client/pom.xml b/client/pom.xml
index aca7747..d447830 100644
--- a/client/pom.xml
+++ b/client/pom.xml
@@ -258,6 +258,11 @@
     </dependency>
     <dependency>
       <groupId>org.apache.cloudstack</groupId>
+      <artifactId>cloud-framework-quota</artifactId>
+      <version>${project.version}</version>
+    </dependency>
+    <dependency>
+      <groupId>org.apache.cloudstack</groupId>
       <artifactId>cloud-framework-rest</artifactId>
       <version>${project.version}</version>
     </dependency>
@@ -366,6 +371,11 @@
       <artifactId>cloud-plugin-network-globodns</artifactId>
       <version>${project.version}</version>
     </dependency>
+    <dependency>
+      <groupId>org.apache.cloudstack</groupId>
+      <artifactId>cloud-plugin-database-quota</artifactId>
+      <version>${project.version}</version>
+    </dependency>
   </dependencies>
   <build>
     <plugins>

http://git-wip-us.apache.org/repos/asf/cloudstack/blob/987fcbd4/client/tomcatconf/commands.properties.in
----------------------------------------------------------------------
diff --git a/client/tomcatconf/commands.properties.in b/client/tomcatconf/commands.properties.in
index b40841b..485abea 100644
--- a/client/tomcatconf/commands.properties.in
+++ b/client/tomcatconf/commands.properties.in
@@ -787,3 +787,14 @@ addGloboDnsHost=1
 ### volume/template post upload
 getUploadParamsForVolume=15
 getUploadParamsForTemplate=15
+
+### Quota Service
+quotaStatement=15
+quotaBalance=15
+quotaSummary=15
+quotaUpdate=1
+quotaTariffList=15
+quotaTariffUpdate=1
+quotaCredits=1
+quotaEmailTemplateList=1
+quotaEmailTemplateUpdate=1

http://git-wip-us.apache.org/repos/asf/cloudstack/blob/987fcbd4/engine/schema/src/com/cloud/upgrade/dao/Upgrade452to460.java
----------------------------------------------------------------------
diff --git a/engine/schema/src/com/cloud/upgrade/dao/Upgrade452to460.java b/engine/schema/src/com/cloud/upgrade/dao/Upgrade452to460.java
index 524ee2e..6b78a7e 100644
--- a/engine/schema/src/com/cloud/upgrade/dao/Upgrade452to460.java
+++ b/engine/schema/src/com/cloud/upgrade/dao/Upgrade452to460.java
@@ -340,4 +340,4 @@ public class Upgrade452to460 implements DbUpgrade {
         }
         s_logger.debug("Updating System Vm Template IDs Complete");
     }
-}
\ No newline at end of file
+}

http://git-wip-us.apache.org/repos/asf/cloudstack/blob/987fcbd4/engine/schema/src/com/cloud/upgrade/dao/Upgrade461to470.java
----------------------------------------------------------------------
diff --git a/engine/schema/src/com/cloud/upgrade/dao/Upgrade461to470.java b/engine/schema/src/com/cloud/upgrade/dao/Upgrade461to470.java
index 4868680..8dbbdb2 100644
--- a/engine/schema/src/com/cloud/upgrade/dao/Upgrade461to470.java
+++ b/engine/schema/src/com/cloud/upgrade/dao/Upgrade461to470.java
@@ -23,6 +23,8 @@ import org.apache.log4j.Logger;
 
 import java.io.File;
 import java.sql.Connection;
+import java.sql.PreparedStatement;
+import java.sql.SQLException;
 
 public class Upgrade461to470 implements DbUpgrade {
     final static Logger s_logger = Logger.getLogger(Upgrade461to470.class);
@@ -51,8 +53,23 @@ public class Upgrade461to470 implements DbUpgrade {
         return new File[] {new File(script)};
     }
 
+    public void alterAddColumnToCloudUsage(final Connection conn) {
+        final String alterTableSql = "ALTER TABLE `cloud_usage`.`cloud_usage` ADD COLUMN `quota_calculated` tinyint(1) DEFAULT 0 NOT NULL COMMENT 'quota calculation status'";
+        try (PreparedStatement pstmt = conn.prepareStatement(alterTableSql)) {
+            pstmt.executeUpdate();
+            s_logger.info("Altered cloud_usage.cloud_usage table and added column quota_calculated");
+        } catch (SQLException e) {
+            if (e.getMessage().contains("quota_calculated")) {
+                s_logger.warn("cloud_usage.cloud_usage table already has a column called quota_calculated");
+            } else {
+                throw new CloudRuntimeException("Unable to create column quota_calculated in table cloud_usage.cloud_usage", e);
+            }
+        }
+    }
+
     @Override
     public void performDataMigration(Connection conn) {
+        alterAddColumnToCloudUsage(conn);
     }
 
     @Override

http://git-wip-us.apache.org/repos/asf/cloudstack/blob/987fcbd4/engine/schema/src/com/cloud/usage/UsageVO.java
----------------------------------------------------------------------
diff --git a/engine/schema/src/com/cloud/usage/UsageVO.java b/engine/schema/src/com/cloud/usage/UsageVO.java
index c46abb3..cc90d71 100644
--- a/engine/schema/src/com/cloud/usage/UsageVO.java
+++ b/engine/schema/src/com/cloud/usage/UsageVO.java
@@ -103,6 +103,17 @@ public class UsageVO implements Usage, InternalIdentity {
     @Temporal(value = TemporalType.TIMESTAMP)
     private Date endDate = null;
 
+    @Column(name = "quota_calculated")
+    private Integer quotaCalculated = 0;
+
+    public Integer getQuotaCalculated() {
+        return quotaCalculated;
+    }
+
+    public void setQuotaCalculated(Integer quotaCalculated) {
+        this.quotaCalculated = quotaCalculated;
+    }
+
     public UsageVO() {
     }
 
@@ -121,8 +132,8 @@ public class UsageVO implements Usage, InternalIdentity {
         this.templateId = templateId;
         this.usageId = usageId;
         this.size = size;
-        this.startDate = startDate;
-        this.endDate = endDate;
+        this.startDate = startDate  == null ? null : new Date(startDate.getTime());
+        this.endDate = endDate  == null ? null : new Date(endDate.getTime());
     }
 
     public UsageVO(Long zoneId, Long accountId, Long domainId, String description, String usageDisplay, int usageType, Double rawUsage, Long vmId, String vmName,
@@ -141,8 +152,8 @@ public class UsageVO implements Usage, InternalIdentity {
         this.usageId = usageId;
         this.size = size;
         this.virtualSize = virtualSize;
-        this.startDate = startDate;
-        this.endDate = endDate;
+        this.startDate = startDate  == null ? null : new Date(startDate.getTime());
+        this.endDate = endDate  == null ? null : new Date(endDate.getTime());
     }
 
     public UsageVO(Long zoneId, Long accountId, Long domainId, String description, String usageDisplay, int usageType, Double rawUsage, Long usageId, String type,
@@ -157,8 +168,8 @@ public class UsageVO implements Usage, InternalIdentity {
         this.usageId = usageId;
         this.type = type;
         this.networkId = networkId;
-        this.startDate = startDate;
-        this.endDate = endDate;
+        this.startDate = startDate  == null ? null : new Date(startDate.getTime());
+        this.endDate = endDate  == null ? null : new Date(endDate.getTime());
     }
 
     public UsageVO(Long zoneId, Long accountId, Long domainId, String description, String usageDisplay, int usageType, Double rawUsage, Long vmId, String vmName,
@@ -176,8 +187,8 @@ public class UsageVO implements Usage, InternalIdentity {
         this.templateId = templateId;
         this.usageId = usageId;
         this.type = type;
-        this.startDate = startDate;
-        this.endDate = endDate;
+        this.startDate = startDate  == null ? null : new Date(startDate.getTime());
+        this.endDate = endDate  == null ? null : new Date(endDate.getTime());
     }
 
     public UsageVO(Long zoneId, Long accountId, Long domainId, String description, String usageDisplay, int usageType, Double rawUsage, Long vmId, String vmName,
@@ -198,8 +209,8 @@ public class UsageVO implements Usage, InternalIdentity {
         this.templateId = templateId;
         this.usageId = usageId;
         this.type = type;
-        this.startDate = startDate;
-        this.endDate = endDate;
+        this.startDate = startDate  == null ? null : new Date(startDate.getTime());
+        this.endDate = endDate  == null ? null : new Date(endDate.getTime());
     }
 
     //IPAddress Usage
@@ -215,8 +226,8 @@ public class UsageVO implements Usage, InternalIdentity {
         this.usageId = usageId;
         this.size = size;
         this.type = type;
-        this.startDate = startDate;
-        this.endDate = endDate;
+        this.startDate = startDate  == null ? null : new Date(startDate.getTime());
+        this.endDate = endDate  == null ? null : new Date(endDate.getTime());
     }
 
     @Override
@@ -321,11 +332,55 @@ public class UsageVO implements Usage, InternalIdentity {
 
     @Override
     public Date getStartDate() {
-        return startDate;
+        return startDate  == null ? null : new Date(startDate.getTime());
     }
 
     @Override
     public Date getEndDate() {
-        return endDate;
+        return endDate  == null ? null : new Date(endDate.getTime());
+    }
+
+    public void setId(Long id) {
+        this.id = id;
+    }
+
+    public void setType(String type) {
+        this.type = type;
+    }
+
+    public void setStartDate(Date startDate) {
+        this.startDate = startDate  == null ? null : new Date(startDate.getTime());
+    }
+
+    public void setEndDate(Date endDate) {
+        this.endDate = endDate  == null ? null : new Date(endDate.getTime());
+    }
+
+    public void setDomainId(Long domainId) {
+        this.domainId = domainId;
+    }
+
+    public void setAccountId(Long accountId) {
+        this.accountId = accountId;
+    }
+
+    public void setZoneId(Long zoneId) {
+        this.zoneId = zoneId;
+    }
+
+    public void setUsageType(int usageType) {
+        this.usageType = usageType;
+    }
+
+    public void setRawUsage(Double rawUsage) {
+        this.rawUsage = rawUsage;
+    }
+
+    public void setSize(Long size) {
+        this.size = size;
+    }
+
+    public void setVirtualSize(Long virtualSize) {
+        this.virtualSize = virtualSize;
     }
 }

http://git-wip-us.apache.org/repos/asf/cloudstack/blob/987fcbd4/engine/schema/src/com/cloud/usage/dao/UsageDao.java
----------------------------------------------------------------------
diff --git a/engine/schema/src/com/cloud/usage/dao/UsageDao.java b/engine/schema/src/com/cloud/usage/dao/UsageDao.java
index f9e9d23..4822dd6 100644
--- a/engine/schema/src/com/cloud/usage/dao/UsageDao.java
+++ b/engine/schema/src/com/cloud/usage/dao/UsageDao.java
@@ -55,4 +55,8 @@ public interface UsageDao extends GenericDao<UsageVO, Long> {
     void saveUsageRecords(List<UsageVO> usageRecords);
 
     void removeOldUsageRecords(int days);
+
+    UsageVO persistUsage(final UsageVO usage);
+
+    Pair<List<? extends UsageVO>, Integer> getUsageRecordsPendingQuotaAggregation(long accountId, long domainId);
 }

http://git-wip-us.apache.org/repos/asf/cloudstack/blob/987fcbd4/engine/schema/src/com/cloud/usage/dao/UsageDaoImpl.java
----------------------------------------------------------------------
diff --git a/engine/schema/src/com/cloud/usage/dao/UsageDaoImpl.java b/engine/schema/src/com/cloud/usage/dao/UsageDaoImpl.java
index 69ea0fb..9c9ab0b 100644
--- a/engine/schema/src/com/cloud/usage/dao/UsageDaoImpl.java
+++ b/engine/schema/src/com/cloud/usage/dao/UsageDaoImpl.java
@@ -24,9 +24,14 @@ import com.cloud.utils.DateUtil;
 import com.cloud.utils.Pair;
 import com.cloud.utils.db.Filter;
 import com.cloud.utils.db.GenericDaoBase;
+import com.cloud.utils.db.QueryBuilder;
 import com.cloud.utils.db.SearchCriteria;
+import com.cloud.utils.db.Transaction;
+import com.cloud.utils.db.TransactionCallback;
 import com.cloud.utils.db.TransactionLegacy;
+import com.cloud.utils.db.TransactionStatus;
 import com.cloud.utils.exception.CloudRuntimeException;
+
 import org.apache.log4j.Logger;
 import org.springframework.stereotype.Component;
 
@@ -45,29 +50,24 @@ public class UsageDaoImpl extends GenericDaoBase<UsageVO, Long> implements Usage
     private static final String DELETE_ALL_BY_ACCOUNTID = "DELETE FROM cloud_usage WHERE account_id = ?";
     private static final String DELETE_ALL_BY_INTERVAL = "DELETE FROM cloud_usage WHERE end_date < DATE_SUB(CURRENT_DATE(), INTERVAL ? DAY)";
     private static final String INSERT_ACCOUNT = "INSERT INTO cloud_usage.account (id, account_name, type, domain_id, removed, cleanup_needed) VALUES (?,?,?,?,?,?)";
-    private static final String INSERT_USER_STATS =
-            "INSERT INTO cloud_usage.user_statistics (id, data_center_id, account_id, public_ip_address, device_id, device_type, network_id, net_bytes_received,"
-                    + " net_bytes_sent, current_bytes_received, current_bytes_sent, agg_bytes_received, agg_bytes_sent) VALUES (?,?,?,?,?,?,?,?,?,?, ?, ?, ?)";
+    private static final String INSERT_USER_STATS = "INSERT INTO cloud_usage.user_statistics (id, data_center_id, account_id, public_ip_address, device_id, device_type, network_id, net_bytes_received,"
+            + " net_bytes_sent, current_bytes_received, current_bytes_sent, agg_bytes_received, agg_bytes_sent) VALUES (?,?,?,?,?,?,?,?,?,?, ?, ?, ?)";
 
     private static final String UPDATE_ACCOUNT = "UPDATE cloud_usage.account SET account_name=?, removed=? WHERE id=?";
-    private static final String UPDATE_USER_STATS =
-            "UPDATE cloud_usage.user_statistics SET net_bytes_received=?, net_bytes_sent=?, current_bytes_received=?, current_bytes_sent=?, agg_bytes_received=?, agg_bytes_sent=? WHERE id=?";
+    private static final String UPDATE_USER_STATS = "UPDATE cloud_usage.user_statistics SET net_bytes_received=?, net_bytes_sent=?, current_bytes_received=?, current_bytes_sent=?, agg_bytes_received=?, agg_bytes_sent=? WHERE id=?";
 
     private static final String GET_LAST_ACCOUNT = "SELECT id FROM cloud_usage.account ORDER BY id DESC LIMIT 1";
     private static final String GET_LAST_USER_STATS = "SELECT id FROM cloud_usage.user_statistics ORDER BY id DESC LIMIT 1";
     private static final String GET_PUBLIC_TEMPLATES_BY_ACCOUNTID = "SELECT id FROM cloud.vm_template WHERE account_id = ? AND public = '1' AND removed IS NULL";
 
     private static final String GET_LAST_VM_DISK_STATS = "SELECT id FROM cloud_usage.vm_disk_statistics ORDER BY id DESC LIMIT 1";
-    private static final String INSERT_VM_DISK_STATS =
-            "INSERT INTO cloud_usage.vm_disk_statistics (id, data_center_id, account_id, vm_id, volume_id, net_io_read, net_io_write, current_io_read, "
-                    + "current_io_write, agg_io_read, agg_io_write, net_bytes_read, net_bytes_write, current_bytes_read, current_bytes_write, agg_bytes_read, agg_bytes_write) "
-                    + " VALUES (?,?,?,?,?,?,?,?,?,?, ?, ?, ?, ?,?, ?, ?)";
-    private static final String UPDATE_VM_DISK_STATS =
-            "UPDATE cloud_usage.vm_disk_statistics SET net_io_read=?, net_io_write=?, current_io_read=?, current_io_write=?, agg_io_read=?, agg_io_write=?, "
-                    + "net_bytes_read=?, net_bytes_write=?, current_bytes_read=?, current_bytes_write=?, agg_bytes_read=?, agg_bytes_write=?  WHERE id=?";
+    private static final String INSERT_VM_DISK_STATS = "INSERT INTO cloud_usage.vm_disk_statistics (id, data_center_id, account_id, vm_id, volume_id, net_io_read, net_io_write, current_io_read, "
+            + "current_io_write, agg_io_read, agg_io_write, net_bytes_read, net_bytes_write, current_bytes_read, current_bytes_write, agg_bytes_read, agg_bytes_write) "
+            + " VALUES (?,?,?,?,?,?,?,?,?,?, ?, ?, ?, ?,?, ?, ?)";
+    private static final String UPDATE_VM_DISK_STATS = "UPDATE cloud_usage.vm_disk_statistics SET net_io_read=?, net_io_write=?, current_io_read=?, current_io_write=?, agg_io_read=?, agg_io_write=?, "
+            + "net_bytes_read=?, net_bytes_write=?, current_bytes_read=?, current_bytes_write=?, agg_bytes_read=?, agg_bytes_write=?  WHERE id=?";
     private static final String INSERT_USAGE_RECORDS = "INSERT INTO cloud_usage.cloud_usage (zone_id, account_id, domain_id, description, usage_display, "
-            +
-            "usage_type, raw_usage, vm_instance_id, vm_name, offering_id, template_id, "
+            + "usage_type, raw_usage, vm_instance_id, vm_name, offering_id, template_id, "
             + "usage_id, type, size, network_id, start_date, end_date, virtual_size) VALUES (?,?,?,?,?,?,?,?,?, ?, ?, ?,?,?,?,?,?,?)";
 
     protected final static TimeZone s_gmtTimeZone = TimeZone.getTimeZone("GMT");
@@ -213,7 +213,7 @@ public class UsageDaoImpl extends GenericDaoBase<UsageVO, Long> implements Usage
             txn.start();
             String sql = UPDATE_USER_STATS;
             PreparedStatement pstmt = null;
-            pstmt = txn.prepareAutoCloseStatement(sql);  // in reality I just want CLOUD_USAGE dataSource connection
+            pstmt = txn.prepareAutoCloseStatement(sql); // in reality I just want CLOUD_USAGE dataSource connection
             for (UserStatisticsVO userStat : userStats) {
                 pstmt.setLong(1, userStat.getNetBytesReceived());
                 pstmt.setLong(2, userStat.getNetBytesSent());
@@ -310,7 +310,7 @@ public class UsageDaoImpl extends GenericDaoBase<UsageVO, Long> implements Usage
             txn.start();
             String sql = UPDATE_VM_DISK_STATS;
             PreparedStatement pstmt = null;
-            pstmt = txn.prepareAutoCloseStatement(sql);  // in reality I just want CLOUD_USAGE dataSource connection
+            pstmt = txn.prepareAutoCloseStatement(sql); // in reality I just want CLOUD_USAGE dataSource connection
             for (VmDiskStatisticsVO vmDiskStat : vmDiskStats) {
                 pstmt.setLong(1, vmDiskStat.getNetIORead());
                 pstmt.setLong(2, vmDiskStat.getNetIOWrite());
@@ -467,4 +467,40 @@ public class UsageDaoImpl extends GenericDaoBase<UsageVO, Long> implements Usage
             txn.close();
         }
     }
+
+    public UsageVO persistUsage(final UsageVO usage) {
+        return Transaction.execute(TransactionLegacy.USAGE_DB, new TransactionCallback<UsageVO>() {
+            @Override
+            public UsageVO doInTransaction(final TransactionStatus status) {
+                return persist(usage);
+            }
+        });
+    }
+
+    public Pair<List<? extends UsageVO>, Integer> getUsageRecordsPendingQuotaAggregation(final long accountId, final long domainId) {
+        if (s_logger.isDebugEnabled()) {
+            s_logger.debug("Getting usage records for account: " + accountId + ", domainId: " + domainId);
+        }
+        return Transaction.execute(TransactionLegacy.USAGE_DB, new TransactionCallback<Pair<List<? extends UsageVO>, Integer>>() {
+            @Override
+            public Pair<List<? extends UsageVO>, Integer> doInTransaction(final TransactionStatus status) {
+                Pair<List<UsageVO>, Integer> usageRecords = new Pair<List<UsageVO>, Integer>(new ArrayList<UsageVO>(), 0);
+                Filter usageFilter = new Filter(UsageVO.class, "startDate", true, 0L, Long.MAX_VALUE);
+                QueryBuilder<UsageVO> qb = QueryBuilder.create(UsageVO.class);
+                if (accountId != -1) {
+                    qb.and(qb.entity().getAccountId(), SearchCriteria.Op.EQ, accountId);
+                }
+                if (domainId != -1) {
+                    qb.and(qb.entity().getDomainId(), SearchCriteria.Op.EQ, domainId);
+                }
+                qb.and(qb.entity().getQuotaCalculated(), SearchCriteria.Op.NEQ, 1);
+                qb.and(qb.entity().getRawUsage(), SearchCriteria.Op.GT, 0);
+                if (s_logger.isDebugEnabled()) {
+                    s_logger.debug("Getting usage records" + usageFilter.getOrderBy());
+                }
+                usageRecords = searchAndCountAllRecords(qb.create(), usageFilter);
+                return new Pair<List<? extends UsageVO>, Integer>(usageRecords.first(), usageRecords.second());
+            }
+        });
+    }
 }

http://git-wip-us.apache.org/repos/asf/cloudstack/blob/987fcbd4/framework/db/src/com/cloud/utils/db/Transaction.java
----------------------------------------------------------------------
diff --git a/framework/db/src/com/cloud/utils/db/Transaction.java b/framework/db/src/com/cloud/utils/db/Transaction.java
index dd91a96..c6a491a 100644
--- a/framework/db/src/com/cloud/utils/db/Transaction.java
+++ b/framework/db/src/com/cloud/utils/db/Transaction.java
@@ -35,18 +35,11 @@ public class Transaction {
         if (currentTxn != null) {
             databaseId = currentTxn.getDatabaseId();
         }
-        TransactionLegacy txn = TransactionLegacy.open(name, databaseId, false);
-        try {
-//            if (txn.dbTxnStarted()){
-//                String warnMsg = "Potential Wrong Usage: TRANSACTION.EXECUTE IS WRAPPED INSIDE ANOTHER DB TRANSACTION!";
-//                s_logger.warn(warnMsg, new CloudRuntimeException(warnMsg));
-//            }
+        try (final TransactionLegacy txn = TransactionLegacy.open(name, databaseId, false)) {
             txn.start();
             T result = callback.doInTransaction(STATUS);
             txn.commit();
             return result;
-        } finally {
-            txn.close();
         }
     }
 
@@ -59,4 +52,28 @@ public class Transaction {
         });
     }
 
+    @SuppressWarnings("deprecation")
+    public static <T, E extends Throwable> T execute(final short databaseId, TransactionCallbackWithException<T, E> callback) throws E {
+        String name = "tx-" + counter.incrementAndGet();
+        TransactionLegacy currentTxn = TransactionLegacy.currentTxn(false);
+        short outer_txn_databaseId = (currentTxn != null ? currentTxn.getDatabaseId() : databaseId);
+        try (final TransactionLegacy txn = TransactionLegacy.open(name, databaseId, true)) {
+            txn.start();
+            T result = callback.doInTransaction(STATUS);
+            txn.commit();
+            return result;
+        } finally {
+            TransactionLegacy.open(outer_txn_databaseId).close();
+        }
+    }
+
+    public static <T> T execute(final short databaseId, final TransactionCallback<T> callback) {
+        return execute(databaseId, new TransactionCallbackWithException<T, RuntimeException>() {
+            @Override
+            public T doInTransaction(TransactionStatus status) throws RuntimeException {
+                return callback.doInTransaction(status);
+            }
+        });
+    }
+
 }

http://git-wip-us.apache.org/repos/asf/cloudstack/blob/987fcbd4/framework/pom.xml
----------------------------------------------------------------------
diff --git a/framework/pom.xml b/framework/pom.xml
index 1b4e17e..3cfc6d0 100644
--- a/framework/pom.xml
+++ b/framework/pom.xml
@@ -47,6 +47,7 @@
     <module>rest</module>
     <module>events</module>
     <module>jobs</module>
+    <module>quota</module>
     <module>cluster</module>
     <module>db</module>
     <module>config</module>

http://git-wip-us.apache.org/repos/asf/cloudstack/blob/987fcbd4/framework/quota/pom.xml
----------------------------------------------------------------------
diff --git a/framework/quota/pom.xml b/framework/quota/pom.xml
new file mode 100644
index 0000000..c0ed3c8
--- /dev/null
+++ b/framework/quota/pom.xml
@@ -0,0 +1,73 @@
+<!--
+  Licensed to the Apache Software Foundation (ASF) under one
+  or more contributor license agreements. See the NOTICE file
+  distributed with this work for additional information
+  regarding copyright ownership. The ASF licenses this file
+  to you under the Apache License, Version 2.0 (the
+  "License"); you may not use this file except in compliance
+  with the License. You may obtain a copy of the License at
+
+  http://www.apache.org/licenses/LICENSE-2.0
+
+  Unless required by applicable law or agreed to in writing,
+  software distributed under the License is distributed on an
+  "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+  KIND, either express or implied. See the License for the
+  specific language governing permissions and limitations
+  under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+  <modelVersion>4.0.0</modelVersion>
+  <artifactId>cloud-framework-quota</artifactId>
+  <name>Apache CloudStack Framework - Quota</name>
+  <parent>
+    <groupId>org.apache.cloudstack</groupId>
+    <artifactId>cloudstack-framework</artifactId>
+    <version>4.7.0-SNAPSHOT</version>
+    <relativePath>../pom.xml</relativePath>
+  </parent>
+  <dependencies>
+    <dependency>
+      <groupId>org.apache.cloudstack</groupId>
+      <artifactId>cloud-utils</artifactId>
+      <version>${project.version}</version>
+    </dependency>
+    <dependency>
+      <groupId>org.apache.cloudstack</groupId>
+      <artifactId>cloud-engine-schema</artifactId>
+      <version>${project.version}</version>
+    </dependency>
+    <dependency>
+        <groupId>junit</groupId>
+        <artifactId>junit</artifactId>
+        <version>${cs.junit.version}</version>
+        <scope>test</scope>
+    </dependency>
+    <dependency>
+        <groupId>org.mockito</groupId>
+        <artifactId>mockito-all</artifactId>
+        <version>${cs.mockito.version}</version>
+        <scope>test</scope>
+    </dependency>
+    <dependency>
+        <groupId>org.powermock</groupId>
+        <artifactId>powermock-module-junit4</artifactId>
+        <version>${cs.powermock.version}</version>
+    </dependency>
+    <dependency>
+        <groupId>org.powermock</groupId>
+        <artifactId>powermock-api-mockito</artifactId>
+        <version>${cs.powermock.version}</version>
+        <scope>test</scope>
+    </dependency>
+    <dependency>
+      <groupId>org.apache.commons</groupId>
+      <artifactId>commons-lang3</artifactId>
+      <version>${cs.commons-lang3.version}</version>
+    </dependency>
+    <dependency>
+      <groupId>javax.mail</groupId>
+      <artifactId>mail</artifactId>
+    </dependency>
+  </dependencies>
+</project>

http://git-wip-us.apache.org/repos/asf/cloudstack/blob/987fcbd4/framework/quota/resources/META-INF/cloudstack/quota/spring-framework-quota-context.xml
----------------------------------------------------------------------
diff --git a/framework/quota/resources/META-INF/cloudstack/quota/spring-framework-quota-context.xml b/framework/quota/resources/META-INF/cloudstack/quota/spring-framework-quota-context.xml
new file mode 100644
index 0000000..f7a3acc
--- /dev/null
+++ b/framework/quota/resources/META-INF/cloudstack/quota/spring-framework-quota-context.xml
@@ -0,0 +1,34 @@
+<!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor
+	license agreements. See the NOTICE file distributed with this work for additional
+	information regarding copyright ownership. The ASF licenses this file to
+	you under the Apache License, Version 2.0 (the "License"); you may not use
+	this file except in compliance with the License. You may obtain a copy of
+	the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required
+	by applicable law or agreed to in writing, software distributed under the
+	License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS
+	OF ANY KIND, either express or implied. See the License for the specific
+	language governing permissions and limitations under the License. -->
+<beans xmlns="http://www.springframework.org/schema/beans"
+	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context"
+	xmlns:aop="http://www.springframework.org/schema/aop"
+	xsi:schemaLocation="http://www.springframework.org/schema/beans
+                      http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
+                      http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-3.0.xsd
+                      http://www.springframework.org/schema/context
+                      http://www.springframework.org/schema/context/spring-context-3.0.xsd">
+
+	<bean id="QuotaTariffDao" class="org.apache.cloudstack.quota.dao.QuotaTariffDaoImpl" />
+    <bean id="QuotaAccountDao" class="org.apache.cloudstack.quota.dao.QuotaAccountDaoImpl" />
+	<bean id="QuotaBalanceDao" class="org.apache.cloudstack.quota.dao.QuotaBalanceDaoImpl" />
+	<bean id="QuotaCreditsDao" class="org.apache.cloudstack.quota.dao.QuotaCreditsDaoImpl" />
+	<bean id="QuotaEmailTemplatesDao"
+		class="org.apache.cloudstack.quota.dao.QuotaEmailTemplatesDaoImpl" />
+	<bean id="QuotaUsageDao" class="org.apache.cloudstack.quota.dao.QuotaUsageDaoImpl" />
+    <bean id="ServiceOfferingDao" class="org.apache.cloudstack.quota.dao.ServiceOfferingDaoImpl" />
+    <bean id="UserVmDetailsDao" class="org.apache.cloudstack.quota.dao.UserVmDetailsDaoImpl" />
+
+	<bean id="QuotaManager" class="org.apache.cloudstack.quota.QuotaManagerImpl" />
+    <bean id="QuotaAlertManager" class="org.apache.cloudstack.quota.QuotaAlertManagerImpl" />
+	<bean id="QuotaStatement" class="org.apache.cloudstack.quota.QuotaStatementImpl" />
+
+</beans>

http://git-wip-us.apache.org/repos/asf/cloudstack/blob/987fcbd4/framework/quota/src/org/apache/cloudstack/quota/QuotaAlertManager.java
----------------------------------------------------------------------
diff --git a/framework/quota/src/org/apache/cloudstack/quota/QuotaAlertManager.java b/framework/quota/src/org/apache/cloudstack/quota/QuotaAlertManager.java
new file mode 100644
index 0000000..44204e8
--- /dev/null
+++ b/framework/quota/src/org/apache/cloudstack/quota/QuotaAlertManager.java
@@ -0,0 +1,26 @@
+//Licensed to the Apache Software Foundation (ASF) under one
+//or more contributor license agreements.  See the NOTICE file
+//distributed with this work for additional information
+//regarding copyright ownership.  The ASF licenses this file
+//to you under the Apache License, Version 2.0 (the
+//"License"); you may not use this file except in compliance
+//the License.  You may obtain a copy of the License at
+//
+//http://www.apache.org/licenses/LICENSE-2.0
+//
+//Unless required by applicable law or agreed to in writing,
+//software distributed under the License is distributed on an
+//"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+//KIND, either express or implied.  See the License for the
+//specific language governing permissions and limitations
+//under the License.
+package org.apache.cloudstack.quota;
+
+import com.cloud.utils.component.Manager;
+
+import org.apache.cloudstack.quota.QuotaAlertManagerImpl.DeferredQuotaEmail;
+
+public interface QuotaAlertManager extends Manager {
+    void checkAndSendQuotaAlertEmails();
+    void sendQuotaAlert(DeferredQuotaEmail emailToBeSent);
+}

http://git-wip-us.apache.org/repos/asf/cloudstack/blob/987fcbd4/framework/quota/src/org/apache/cloudstack/quota/QuotaAlertManagerImpl.java
----------------------------------------------------------------------
diff --git a/framework/quota/src/org/apache/cloudstack/quota/QuotaAlertManagerImpl.java b/framework/quota/src/org/apache/cloudstack/quota/QuotaAlertManagerImpl.java
new file mode 100644
index 0000000..a57e0c2
--- /dev/null
+++ b/framework/quota/src/org/apache/cloudstack/quota/QuotaAlertManagerImpl.java
@@ -0,0 +1,418 @@
+//Licensed to the Apache Software Foundation (ASF) under one
+//or more contributor license agreements.  See the NOTICE file
+//distributed with this work for additional information
+//regarding copyright ownership.  The ASF licenses this file
+//to you under the Apache License, Version 2.0 (the
+//"License"); you may not use this file except in compliance
+//with the License.  You may obtain a copy of the License at
+//
+//http://www.apache.org/licenses/LICENSE-2.0
+//
+//Unless required by applicable law or agreed to in writing,
+//software distributed under the License is distributed on an
+//"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+//KIND, either express or implied.  See the License for the
+//specific language governing permissions and limitations
+//under the License.
+package org.apache.cloudstack.quota;
+
+import com.cloud.domain.DomainVO;
+import com.cloud.domain.dao.DomainDao;
+import com.cloud.user.Account;
+import com.cloud.user.Account.State;
+import com.cloud.user.AccountVO;
+import com.cloud.user.UserVO;
+import com.cloud.user.dao.AccountDao;
+import com.cloud.user.dao.UserDao;
+import com.cloud.utils.NumbersUtil;
+import com.cloud.utils.component.ManagerBase;
+import com.cloud.utils.db.TransactionLegacy;
+import com.cloud.utils.exception.CloudRuntimeException;
+import com.google.common.base.Strings;
+import com.sun.mail.smtp.SMTPMessage;
+import com.sun.mail.smtp.SMTPSSLTransport;
+import com.sun.mail.smtp.SMTPTransport;
+import org.apache.cloudstack.framework.config.dao.ConfigurationDao;
+import org.apache.cloudstack.quota.constant.QuotaConfig;
+import org.apache.cloudstack.quota.constant.QuotaConfig.QuotaEmailTemplateTypes;
+import org.apache.cloudstack.quota.dao.QuotaAccountDao;
+import org.apache.cloudstack.quota.dao.QuotaEmailTemplatesDao;
+import org.apache.cloudstack.quota.dao.QuotaUsageDao;
+import org.apache.cloudstack.quota.vo.QuotaAccountVO;
+import org.apache.cloudstack.quota.vo.QuotaEmailTemplatesVO;
+import org.apache.commons.lang3.text.StrSubstitutor;
+import org.apache.log4j.Logger;
+import org.springframework.stereotype.Component;
+
+import javax.ejb.Local;
+import javax.inject.Inject;
+import javax.mail.Authenticator;
+import javax.mail.Message;
+import javax.mail.MessagingException;
+import javax.mail.PasswordAuthentication;
+import javax.mail.Session;
+import javax.mail.URLName;
+import javax.mail.internet.InternetAddress;
+import javax.naming.ConfigurationException;
+import java.io.UnsupportedEncodingException;
+import java.math.BigDecimal;
+import java.util.ArrayList;
+import java.util.Date;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Properties;
+import java.util.concurrent.TimeUnit;
+
+@Component
+@Local(value = QuotaAlertManager.class)
+public class QuotaAlertManagerImpl extends ManagerBase implements QuotaAlertManager {
+    private static final Logger s_logger = Logger.getLogger(QuotaAlertManagerImpl.class);
+
+    @Inject
+    private AccountDao _accountDao;
+    @Inject
+    private QuotaAccountDao _quotaAcc;
+    @Inject
+    private UserDao _userDao;
+    @Inject
+    private DomainDao _domainDao;
+    @Inject
+    private QuotaEmailTemplatesDao _quotaEmailTemplateDao;
+    @Inject
+    private ConfigurationDao _configDao;
+    @Inject
+    private QuotaUsageDao _quotaUsage;
+
+    private EmailQuotaAlert _emailQuotaAlert;
+    private boolean _lockAccountEnforcement = false;
+
+    boolean _smtpDebug = false;
+
+    public QuotaAlertManagerImpl() {
+        super();
+    }
+
+    private void mergeConfigs(Map<String, String> dbParams, Map<String, Object> xmlParams) {
+        for (Map.Entry<String, Object> param : xmlParams.entrySet()) {
+            dbParams.put(param.getKey(), (String)param.getValue());
+        }
+    }
+
+    @Override
+    public boolean configure(String name, Map<String, Object> params) throws ConfigurationException {
+        super.configure(name, params);
+
+        Map<String, String> configs = _configDao.getConfiguration(params);
+
+        if (params != null) {
+            mergeConfigs(configs, params);
+        }
+
+        final String smtpHost = configs.get(QuotaConfig.QuotaSmtpHost.key());
+        int smtpPort = NumbersUtil.parseInt(configs.get(QuotaConfig.QuotaSmtpPort.key()), 25);
+        String useAuthStr = configs.get(QuotaConfig.QuotaSmtpAuthType.key());
+        boolean useAuth = ((useAuthStr != null) && Boolean.parseBoolean(useAuthStr));
+        String smtpUsername = configs.get(QuotaConfig.QuotaSmtpUser.key());
+        String smtpPassword = configs.get(QuotaConfig.QuotaSmtpPassword.key());
+        String emailSender = configs.get(QuotaConfig.QuotaSmtpSender.key());
+        _lockAccountEnforcement = "true".equalsIgnoreCase(configs.get(QuotaConfig.QuotaEnableEnforcement.key()));
+        _emailQuotaAlert = new EmailQuotaAlert(smtpHost, smtpPort, useAuth, smtpUsername, smtpPassword, emailSender, _smtpDebug);
+
+        return true;
+    }
+
+    @Override
+    public boolean start() {
+        if (s_logger.isInfoEnabled()) {
+            s_logger.info("Starting Alert Manager");
+        }
+        return true;
+    }
+
+    @Override
+    public boolean stop() {
+        if (s_logger.isInfoEnabled()) {
+            s_logger.info("Stopping Alert Manager");
+        }
+        return true;
+    }
+
+    @Override
+    public void checkAndSendQuotaAlertEmails() {
+        List<DeferredQuotaEmail> deferredQuotaEmailList = new ArrayList<DeferredQuotaEmail>();
+        final BigDecimal zeroBalance = new BigDecimal(0);
+        for (final QuotaAccountVO quotaAccount : _quotaAcc.listAllQuotaAccount()) {
+            if (s_logger.isDebugEnabled()) {
+                s_logger.debug("checkAndSendQuotaAlertEmails accId=" + quotaAccount.getId());
+            }
+            BigDecimal accountBalance = quotaAccount.getQuotaBalance();
+            Date balanceDate = quotaAccount.getQuotaBalanceDate();
+            Date alertDate = quotaAccount.getQuotaAlertDate();
+            int lockable = quotaAccount.getQuotaEnforce();
+            BigDecimal thresholdBalance = quotaAccount.getQuotaMinBalance();
+            if (accountBalance != null) {
+                AccountVO account = _accountDao.findById(quotaAccount.getId());
+                if (s_logger.isDebugEnabled()) {
+                    s_logger.debug("checkAndSendQuotaAlertEmails: Check id=" + account.getId() + " bal=" + accountBalance + ", alertDate=" + alertDate + ", lockable=" + lockable);
+                }
+                if (accountBalance.compareTo(zeroBalance) < 0) {
+                    if (_lockAccountEnforcement && (lockable == 1)) {
+                        if (account.getType() == Account.ACCOUNT_TYPE_NORMAL) {
+                            s_logger.info("Locking account " + account.getAccountName() + " due to quota < 0.");
+                            lockAccount(account.getId());
+                        }
+                    }
+                    if (alertDate == null || (balanceDate.after(alertDate) && getDifferenceDays(alertDate, new Date()) > 1)) {
+                        s_logger.info("Sending alert " + account.getAccountName() + " due to quota < 0.");
+                        deferredQuotaEmailList.add(new DeferredQuotaEmail(account, quotaAccount, QuotaConfig.QuotaEmailTemplateTypes.QUOTA_EMPTY));
+                    }
+                } else if (accountBalance.compareTo(thresholdBalance) < 0) {
+                    if (alertDate == null || (balanceDate.after(alertDate) && getDifferenceDays(alertDate, new Date()) > 1)) {
+                        s_logger.info("Sending alert " + account.getAccountName() + " due to quota below threshold.");
+                        deferredQuotaEmailList.add(new DeferredQuotaEmail(account, quotaAccount, QuotaConfig.QuotaEmailTemplateTypes.QUOTA_LOW));
+                    }
+                }
+            }
+        }
+
+        for (DeferredQuotaEmail emailToBeSent : deferredQuotaEmailList) {
+            if (s_logger.isDebugEnabled()) {
+                s_logger.debug("checkAndSendQuotaAlertEmails: Attempting to send quota alert email to users of account: " + emailToBeSent.getAccount().getAccountName());
+            }
+            sendQuotaAlert(emailToBeSent);
+        }
+    }
+
+    public void sendQuotaAlert(DeferredQuotaEmail emailToBeSent) {
+        final AccountVO account = emailToBeSent.getAccount();
+        final BigDecimal balance = emailToBeSent.getQuotaBalance();
+        final BigDecimal usage = emailToBeSent.getQuotaUsage();
+        final QuotaConfig.QuotaEmailTemplateTypes emailType = emailToBeSent.getEmailTemplateType();
+
+        final List<QuotaEmailTemplatesVO> emailTemplates = _quotaEmailTemplateDao.listAllQuotaEmailTemplates(emailType.toString());
+        if (emailTemplates != null && emailTemplates.get(0) != null) {
+            final QuotaEmailTemplatesVO emailTemplate = emailTemplates.get(0);
+
+            final DomainVO accountDomain = _domainDao.findByIdIncludingRemoved(account.getDomainId());
+            final List<UserVO> usersInAccount = _userDao.listByAccount(account.getId());
+
+            String userNames = "";
+            final List<String> emailRecipients = new ArrayList<String>();
+            for (UserVO user : usersInAccount) {
+                userNames += String.format("%s <%s>,", user.getUsername(), user.getEmail());
+                emailRecipients.add(user.getEmail());
+            }
+            if (userNames.endsWith(",")) {
+                userNames = userNames.substring(0, userNames.length() - 1);
+            }
+
+            final Map<String, String> optionMap = new HashMap<String, String>();
+            optionMap.put("accountName", account.getAccountName());
+            optionMap.put("accountID", account.getUuid());
+            optionMap.put("accountUsers", userNames);
+            optionMap.put("domainName", accountDomain.getName());
+            optionMap.put("domainID", accountDomain.getUuid());
+            optionMap.put("quotaBalance", QuotaConfig.QuotaCurrencySymbol.value() + " " + balance.toString());
+            if (emailType == QuotaEmailTemplateTypes.QUOTA_STATEMENT) {
+                optionMap.put("quotaUsage", QuotaConfig.QuotaCurrencySymbol.value() + " " + usage.toString());
+            }
+
+            if (s_logger.isDebugEnabled()) {
+                s_logger.debug("accountName" + account.getAccountName() + "accountID" + account.getUuid() + "accountUsers" + userNames + "domainName" + accountDomain.getName()
+                        + "domainID" + accountDomain.getUuid());
+            }
+
+            final StrSubstitutor templateEngine = new StrSubstitutor(optionMap);
+            final String subject = templateEngine.replace(emailTemplate.getTemplateSubject());
+            final String body = templateEngine.replace(emailTemplate.getTemplateBody());
+            try {
+                _emailQuotaAlert.sendQuotaAlert(emailRecipients, subject, body);
+                emailToBeSent.sentSuccessfully(_quotaAcc);
+            } catch (Exception e) {
+                s_logger.error(String.format("Unable to send quota alert email (subject=%s; body=%s) to account %s (%s) recipients (%s) due to error (%s)", subject, body,
+                        account.getAccountName(), account.getUuid(), emailRecipients, e));
+                if (s_logger.isDebugEnabled()) {
+                    s_logger.debug("Exception", e);
+                }
+            }
+        } else {
+            s_logger.error(String.format("No quota email template found for type %s, cannot send quota alert email to account %s(%s)", emailType, account.getAccountName(),
+                    account.getUuid()));
+        }
+    }
+
+    public static long getDifferenceDays(Date d1, Date d2) {
+        long diff = d2.getTime() - d1.getTime();
+        return TimeUnit.DAYS.convert(diff, TimeUnit.MILLISECONDS);
+    }
+
+    protected boolean lockAccount(long accountId) {
+        final short opendb = TransactionLegacy.currentTxn().getDatabaseId();
+        boolean success = false;
+        try (TransactionLegacy txn = TransactionLegacy.open(TransactionLegacy.CLOUD_DB)) {
+            Account account = _accountDao.findById(accountId);
+            if (account != null) {
+                if (account.getState() == State.locked) {
+                    return true; // already locked, no-op
+                } else if (account.getState() == State.enabled) {
+                    AccountVO acctForUpdate = _accountDao.createForUpdate();
+                    acctForUpdate.setState(State.locked);
+                    success = _accountDao.update(Long.valueOf(accountId), acctForUpdate);
+                } else {
+                    if (s_logger.isInfoEnabled()) {
+                        s_logger.info("Attempting to lock a non-enabled account, current state is " + account.getState() + " (accountId: " + accountId + "), locking failed.");
+                    }
+                }
+            } else {
+                s_logger.warn("Failed to lock account " + accountId + ", account not found.");
+            }
+        } catch (Exception e) {
+            s_logger.error("Exception occured while locking account by Quota Alert Manager", e);
+            throw e;
+        } finally {
+            TransactionLegacy.open(opendb).close();
+        }
+        return success;
+    }
+
+    public static class DeferredQuotaEmail {
+        private AccountVO account;
+        private QuotaAccountVO quotaAccount;
+        private QuotaConfig.QuotaEmailTemplateTypes emailTemplateType;
+        private BigDecimal quotaUsage;
+
+        public DeferredQuotaEmail(AccountVO account, QuotaAccountVO quotaAccount, BigDecimal quotaUsage, QuotaConfig.QuotaEmailTemplateTypes emailTemplateType) {
+            this.account = account;
+            this.quotaAccount = quotaAccount;
+            this.emailTemplateType = emailTemplateType;
+            this.quotaUsage = quotaUsage;
+        }
+
+        public DeferredQuotaEmail(AccountVO account, QuotaAccountVO quotaAccount, QuotaConfig.QuotaEmailTemplateTypes emailTemplateType) {
+            this.account = account;
+            this.quotaAccount = quotaAccount;
+            this.emailTemplateType = emailTemplateType;
+            this.quotaUsage = new BigDecimal(-1);
+        }
+
+        public AccountVO getAccount() {
+            return account;
+        }
+
+        public BigDecimal getQuotaBalance() {
+            return quotaAccount.getQuotaBalance();
+        }
+
+        public BigDecimal getQuotaUsage() {
+            return quotaUsage;
+        }
+
+        public Date getSendDate() {
+            if (emailTemplateType == QuotaEmailTemplateTypes.QUOTA_STATEMENT) {
+                return quotaAccount.getLastStatementDate();
+            } else {
+                return quotaAccount.getQuotaAlertDate();
+            }
+        }
+
+        public QuotaConfig.QuotaEmailTemplateTypes getEmailTemplateType() {
+            return emailTemplateType;
+        }
+
+        public void sentSuccessfully(final QuotaAccountDao quotaAccountDao) {
+            if (emailTemplateType == QuotaEmailTemplateTypes.QUOTA_STATEMENT) {
+                quotaAccount.setLastStatementDate(new Date());
+            } else {
+                quotaAccount.setQuotaAlertDate(new Date());
+                quotaAccount.setQuotaAlertType(emailTemplateType.ordinal());
+            }
+            quotaAccountDao.updateQuotaAccount(quotaAccount.getAccountId(), quotaAccount);
+        }
+    };
+
+    static class EmailQuotaAlert {
+        private final Session _smtpSession;
+        private final String _smtpHost;
+        private final int _smtpPort;
+        private final boolean _smtpUseAuth;
+        private final String _smtpUsername;
+        private final String _smtpPassword;
+        private final String _emailSender;
+
+        public EmailQuotaAlert(String smtpHost, int smtpPort, boolean smtpUseAuth, final String smtpUsername, final String smtpPassword, String emailSender, boolean smtpDebug) {
+            _smtpHost = smtpHost;
+            _smtpPort = smtpPort;
+            _smtpUseAuth = smtpUseAuth;
+            _smtpUsername = smtpUsername;
+            _smtpPassword = smtpPassword;
+            _emailSender = emailSender;
+
+            if (!Strings.isNullOrEmpty(_smtpHost)) {
+                Properties smtpProps = new Properties();
+                smtpProps.put("mail.smtp.host", smtpHost);
+                smtpProps.put("mail.smtp.port", smtpPort);
+                smtpProps.put("mail.smtp.auth", "" + smtpUseAuth);
+                if (smtpUsername != null) {
+                    smtpProps.put("mail.smtp.user", smtpUsername);
+                }
+
+                smtpProps.put("mail.smtps.host", smtpHost);
+                smtpProps.put("mail.smtps.port", smtpPort);
+                smtpProps.put("mail.smtps.auth", "" + smtpUseAuth);
+                if (!Strings.isNullOrEmpty(smtpUsername)) {
+                    smtpProps.put("mail.smtps.user", smtpUsername);
+                }
+
+                if (!Strings.isNullOrEmpty(smtpUsername) && !Strings.isNullOrEmpty(smtpPassword)) {
+                    _smtpSession = Session.getInstance(smtpProps, new Authenticator() {
+                        @Override
+                        protected PasswordAuthentication getPasswordAuthentication() {
+                            return new PasswordAuthentication(smtpUsername, smtpPassword);
+                        }
+                    });
+                } else {
+                    _smtpSession = Session.getInstance(smtpProps);
+                }
+                _smtpSession.setDebug(smtpDebug);
+            } else {
+                _smtpSession = null;
+            }
+        }
+
+        public void sendQuotaAlert(List<String> emails, String subject, String body) throws MessagingException, UnsupportedEncodingException {
+            if (_smtpSession == null) {
+                throw new CloudRuntimeException("Unable to create smtp session.");
+            }
+            SMTPMessage msg = new SMTPMessage(_smtpSession);
+            msg.setSender(new InternetAddress(_emailSender, _emailSender));
+            msg.setFrom(new InternetAddress(_emailSender, _emailSender));
+
+            for (String email : emails) {
+                if (email != null && !email.isEmpty()) {
+                    try {
+                        InternetAddress address = new InternetAddress(email, email);
+                        msg.addRecipient(Message.RecipientType.TO, address);
+                    } catch (Exception pokemon) {
+                        s_logger.error("Exception in creating address for:" + email, pokemon);
+                    }
+                }
+            }
+
+            msg.setSubject(subject);
+            msg.setSentDate(new Date());
+            msg.setContent(body, "text/html; charset=utf-8");
+            msg.saveChanges();
+
+            SMTPTransport smtpTrans = null;
+            if (_smtpUseAuth) {
+                smtpTrans = new SMTPSSLTransport(_smtpSession, new URLName("smtp", _smtpHost, _smtpPort, null, _smtpUsername, _smtpPassword));
+            } else {
+                smtpTrans = new SMTPTransport(_smtpSession, new URLName("smtp", _smtpHost, _smtpPort, null, _smtpUsername, _smtpPassword));
+            }
+            smtpTrans.connect();
+            smtpTrans.sendMessage(msg, msg.getAllRecipients());
+            smtpTrans.close();
+        }
+    }
+}

http://git-wip-us.apache.org/repos/asf/cloudstack/blob/987fcbd4/framework/quota/src/org/apache/cloudstack/quota/QuotaManager.java
----------------------------------------------------------------------
diff --git a/framework/quota/src/org/apache/cloudstack/quota/QuotaManager.java b/framework/quota/src/org/apache/cloudstack/quota/QuotaManager.java
new file mode 100644
index 0000000..1cda3b2
--- /dev/null
+++ b/framework/quota/src/org/apache/cloudstack/quota/QuotaManager.java
@@ -0,0 +1,25 @@
+//Licensed to the Apache Software Foundation (ASF) under one
+//or more contributor license agreements.  See the NOTICE file
+//distributed with this work for additional information
+//regarding copyright ownership.  The ASF licenses this file
+//to you under the Apache License, Version 2.0 (the
+//"License"); you may not use this file except in compliance
+//the License.  You may obtain a copy of the License at
+//
+//http://www.apache.org/licenses/LICENSE-2.0
+//
+//Unless required by applicable law or agreed to in writing,
+//software distributed under the License is distributed on an
+//"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+//KIND, either express or implied.  See the License for the
+//specific language governing permissions and limitations
+//under the License.
+package org.apache.cloudstack.quota;
+
+import com.cloud.utils.component.Manager;
+
+public interface QuotaManager extends Manager {
+
+    boolean calculateQuotaUsage();
+
+}


Mime
View raw message