Skip to content

Commit be70d0c

Browse files
hindenbughindenbug
committed
csv read into db mongo
0 parents  commit be70d0c

File tree

7 files changed

+279
-0
lines changed

7 files changed

+279
-0
lines changed

‎.gitignore

+9
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
/target
2+
/classes
3+
/checkouts
4+
pom.xml
5+
pom.xml.asc
6+
*.jar
7+
*.class
8+
/.lein-*
9+
/.nrepl-port

‎LICENSE

+214
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,214 @@
1+
THE ACCOMPANYING PROGRAM IS PROVIDED UNDER THE TERMS OF THIS ECLIPSE PUBLIC
2+
LICENSE ("AGREEMENT"). ANY USE, REPRODUCTION OR DISTRIBUTION OF THE PROGRAM
3+
CONSTITUTES RECIPIENT'S ACCEPTANCE OF THIS AGREEMENT.
4+
5+
1. DEFINITIONS
6+
7+
"Contribution" means:
8+
9+
a) in the case of the initial Contributor, the initial code and
10+
documentation distributed under this Agreement, and
11+
12+
b) in the case of each subsequent Contributor:
13+
14+
i) changes to the Program, and
15+
16+
ii) additions to the Program;
17+
18+
where such changes and/or additions to the Program originate from and are
19+
distributed by that particular Contributor. A Contribution 'originates' from
20+
a Contributor if it was added to the Program by such Contributor itself or
21+
anyone acting on such Contributor's behalf. Contributions do not include
22+
additions to the Program which: (i) are separate modules of software
23+
distributed in conjunction with the Program under their own license
24+
agreement, and (ii) are not derivative works of the Program.
25+
26+
"Contributor" means any person or entity that distributes the Program.
27+
28+
"Licensed Patents" mean patent claims licensable by a Contributor which are
29+
necessarily infringed by the use or sale of its Contribution alone or when
30+
combined with the Program.
31+
32+
"Program" means the Contributions distributed in accordance with this
33+
Agreement.
34+
35+
"Recipient" means anyone who receives the Program under this Agreement,
36+
including all Contributors.
37+
38+
2. GRANT OF RIGHTS
39+
40+
a) Subject to the terms of this Agreement, each Contributor hereby grants
41+
Recipient a non-exclusive, worldwide, royalty-free copyright license to
42+
reproduce, prepare derivative works of, publicly display, publicly perform,
43+
distribute and sublicense the Contribution of such Contributor, if any, and
44+
such derivative works, in source code and object code form.
45+
46+
b) Subject to the terms of this Agreement, each Contributor hereby grants
47+
Recipient a non-exclusive, worldwide, royalty-free patent license under
48+
Licensed Patents to make, use, sell, offer to sell, import and otherwise
49+
transfer the Contribution of such Contributor, if any, in source code and
50+
object code form. This patent license shall apply to the combination of the
51+
Contribution and the Program if, at the time the Contribution is added by the
52+
Contributor, such addition of the Contribution causes such combination to be
53+
covered by the Licensed Patents. The patent license shall not apply to any
54+
other combinations which include the Contribution. No hardware per se is
55+
licensed hereunder.
56+
57+
c) Recipient understands that although each Contributor grants the licenses
58+
to its Contributions set forth herein, no assurances are provided by any
59+
Contributor that the Program does not infringe the patent or other
60+
intellectual property rights of any other entity. Each Contributor disclaims
61+
any liability to Recipient for claims brought by any other entity based on
62+
infringement of intellectual property rights or otherwise. As a condition to
63+
exercising the rights and licenses granted hereunder, each Recipient hereby
64+
assumes sole responsibility to secure any other intellectual property rights
65+
needed, if any. For example, if a third party patent license is required to
66+
allow Recipient to distribute the Program, it is Recipient's responsibility
67+
to acquire that license before distributing the Program.
68+
69+
d) Each Contributor represents that to its knowledge it has sufficient
70+
copyright rights in its Contribution, if any, to grant the copyright license
71+
set forth in this Agreement.
72+
73+
3. REQUIREMENTS
74+
75+
A Contributor may choose to distribute the Program in object code form under
76+
its own license agreement, provided that:
77+
78+
a) it complies with the terms and conditions of this Agreement; and
79+
80+
b) its license agreement:
81+
82+
i) effectively disclaims on behalf of all Contributors all warranties and
83+
conditions, express and implied, including warranties or conditions of title
84+
and non-infringement, and implied warranties or conditions of merchantability
85+
and fitness for a particular purpose;
86+
87+
ii) effectively excludes on behalf of all Contributors all liability for
88+
damages, including direct, indirect, special, incidental and consequential
89+
damages, such as lost profits;
90+
91+
iii) states that any provisions which differ from this Agreement are offered
92+
by that Contributor alone and not by any other party; and
93+
94+
iv) states that source code for the Program is available from such
95+
Contributor, and informs licensees how to obtain it in a reasonable manner on
96+
or through a medium customarily used for software exchange.
97+
98+
When the Program is made available in source code form:
99+
100+
a) it must be made available under this Agreement; and
101+
102+
b) a copy of this Agreement must be included with each copy of the Program.
103+
104+
Contributors may not remove or alter any copyright notices contained within
105+
the Program.
106+
107+
Each Contributor must identify itself as the originator of its Contribution,
108+
if any, in a manner that reasonably allows subsequent Recipients to identify
109+
the originator of the Contribution.
110+
111+
4. COMMERCIAL DISTRIBUTION
112+
113+
Commercial distributors of software may accept certain responsibilities with
114+
respect to end users, business partners and the like. While this license is
115+
intended to facilitate the commercial use of the Program, the Contributor who
116+
includes the Program in a commercial product offering should do so in a
117+
manner which does not create potential liability for other Contributors.
118+
Therefore, if a Contributor includes the Program in a commercial product
119+
offering, such Contributor ("Commercial Contributor") hereby agrees to defend
120+
and indemnify every other Contributor ("Indemnified Contributor") against any
121+
losses, damages and costs (collectively "Losses") arising from claims,
122+
lawsuits and other legal actions brought by a third party against the
123+
Indemnified Contributor to the extent caused by the acts or omissions of such
124+
Commercial Contributor in connection with its distribution of the Program in
125+
a commercial product offering. The obligations in this section do not apply
126+
to any claims or Losses relating to any actual or alleged intellectual
127+
property infringement. In order to qualify, an Indemnified Contributor must:
128+
a) promptly notify the Commercial Contributor in writing of such claim, and
129+
b) allow the Commercial Contributor tocontrol, and cooperate with the
130+
Commercial Contributor in, the defense and any related settlement
131+
negotiations. The Indemnified Contributor may participate in any such claim
132+
at its own expense.
133+
134+
For example, a Contributor might include the Program in a commercial product
135+
offering, Product X. That Contributor is then a Commercial Contributor. If
136+
that Commercial Contributor then makes performance claims, or offers
137+
warranties related to Product X, those performance claims and warranties are
138+
such Commercial Contributor's responsibility alone. Under this section, the
139+
Commercial Contributor would have to defend claims against the other
140+
Contributors related to those performance claims and warranties, and if a
141+
court requires any other Contributor to pay any damages as a result, the
142+
Commercial Contributor must pay those damages.
143+
144+
5. NO WARRANTY
145+
146+
EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, THE PROGRAM IS PROVIDED ON
147+
AN "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, EITHER
148+
EXPRESS OR IMPLIED INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES OR
149+
CONDITIONS OF TITLE, NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A
150+
PARTICULAR PURPOSE. Each Recipient is solely responsible for determining the
151+
appropriateness of using and distributing the Program and assumes all risks
152+
associated with its exercise of rights under this Agreement , including but
153+
not limited to the risks and costs of program errors, compliance with
154+
applicable laws, damage to or loss of data, programs or equipment, and
155+
unavailability or interruption of operations.
156+
157+
6. DISCLAIMER OF LIABILITY
158+
159+
EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, NEITHER RECIPIENT NOR ANY
160+
CONTRIBUTORS SHALL HAVE ANY LIABILITY FOR ANY DIRECT, INDIRECT, INCIDENTAL,
161+
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING WITHOUT LIMITATION
162+
LOST PROFITS), HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
163+
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
164+
ARISING IN ANY WAY OUT OF THE USE OR DISTRIBUTION OF THE PROGRAM OR THE
165+
EXERCISE OF ANY RIGHTS GRANTED HEREUNDER, EVEN IF ADVISED OF THE POSSIBILITY
166+
OF SUCH DAMAGES.
167+
168+
7. GENERAL
169+
170+
If any provision of this Agreement is invalid or unenforceable under
171+
applicable law, it shall not affect the validity or enforceability of the
172+
remainder of the terms of this Agreement, and without further action by the
173+
parties hereto, such provision shall be reformed to the minimum extent
174+
necessary to make such provision valid and enforceable.
175+
176+
If Recipient institutes patent litigation against any entity (including a
177+
cross-claim or counterclaim in a lawsuit) alleging that the Program itself
178+
(excluding combinations of the Program with other software or hardware)
179+
infringes such Recipient's patent(s), then such Recipient's rights granted
180+
under Section 2(b) shall terminate as of the date such litigation is filed.
181+
182+
All Recipient's rights under this Agreement shall terminate if it fails to
183+
comply with any of the material terms or conditions of this Agreement and
184+
does not cure such failure in a reasonable period of time after becoming
185+
aware of such noncompliance. If all Recipient's rights under this Agreement
186+
terminate, Recipient agrees to cease use and distribution of the Program as
187+
soon as reasonably practicable. However, Recipient's obligations under this
188+
Agreement and any licenses granted by Recipient relating to the Program shall
189+
continue and survive.
190+
191+
Everyone is permitted to copy and distribute copies of this Agreement, but in
192+
order to avoid inconsistency the Agreement is copyrighted and may only be
193+
modified in the following manner. The Agreement Steward reserves the right to
194+
publish new versions (including revisions) of this Agreement from time to
195+
time. No one other than the Agreement Steward has the right to modify this
196+
Agreement. The Eclipse Foundation is the initial Agreement Steward. The
197+
Eclipse Foundation may assign the responsibility to serve as the Agreement
198+
Steward to a suitable separate entity. Each new version of the Agreement will
199+
be given a distinguishing version number. The Program (including
200+
Contributions) may always be distributed subject to the version of the
201+
Agreement under which it was received. In addition, after a new version of
202+
the Agreement is published, Contributor may elect to distribute the Program
203+
(including its Contributions) under the new version. Except as expressly
204+
stated in Sections 2(a) and 2(b) above, Recipient receives no rights or
205+
licenses to the intellectual property of any Contributor under this
206+
Agreement, whether expressly, by implication, estoppel or otherwise. All
207+
rights in the Program not expressly granted under this Agreement are
208+
reserved.
209+
210+
This Agreement is governed by the laws of the State of New York and the
211+
intellectual property laws of the United States of America. No party to this
212+
Agreement will bring a legal action under this Agreement more than one year
213+
after the cause of action arose. Each party waives its rights to a jury trial
214+
in any resulting litigation.

‎README.md

+14
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
# import-data
2+
3+
A Clojure library designed to ... well, that part is up to you.
4+
5+
## Usage
6+
7+
FIXME
8+
9+
## License
10+
11+
Copyright © 2014 FIXME
12+
13+
Distributed under the Eclipse Public License either version 1.0 or (at
14+
your option) any later version.

‎doc/intro.md

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# Introduction to import-data
2+
3+
TODO: write [great documentation](http://jacobian.org/writing/what-to-write/)

‎project.clj

+11
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
(defproject import-data "0.1.0-SNAPSHOT"
2+
:description "FIXME: write description"
3+
:url "http://example.com/FIXME"
4+
:license {:name "Eclipse Public License"
5+
:url "http://www.eclipse.org/legal/epl-v10.html"}
6+
:dependencies [[org.clojure/clojure "1.6.0"]
7+
[org.clojure/clojure-contrib "1.2.0"]
8+
[clojure-csv/clojure-csv "2.0.1"]
9+
[com.novemberain/monger "2.0.0"]
10+
[org.clojure/tools.cli "0.3.1"]]
11+
:main import-data.core)

‎src/import_data/core.clj

+21
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
(ns import-data.core
2+
(:require [clojure-csv.core :as csv]
3+
[monger.core :as mg]
4+
[monger.collection :as mc]
5+
[clojure.java.io :as io]
6+
[clojure.tools.cli :refer [cli]])
7+
(:import org.bson.types.ObjectId))
8+
9+
(def db
10+
(mg/get-db (mg/connect) "monger-test"))
11+
12+
(def parse-file
13+
(with-open [file (clojure.java.io/reader (clojure.java.io/resource "data.csv"))]
14+
(doseq [line (line-seq file)]
15+
(let [cat (rand-nth ["100" "200" "300"])
16+
subcat (get (into {} [["100" [101 102 103]] ["200" [201 202 203]] ["300" [301 302 303]]]) cat)]
17+
(mc/insert db "subscriptions" {:user_id line, :category_id (int (read-string cat)), :subcategory_id (int (rand-nth subcat)), :metainfo (take 3 (shuffle ["meta1" "meta2" "meta3" "meta4" "meta5" "meta6" "meta7" "meta8" "meta9"])), :timestamp (new java.util.Date) })
18+
(println line)))))
19+
20+
(defn -main [& args]
21+
(parse-file))

‎test/import_data/core_test.clj

+7
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
(ns import-data.core-test
2+
(:require [clojure.test :refer :all]
3+
[import-data.core :refer :all]))
4+
5+
(deftest a-test
6+
(testing "FIXME, I fail."
7+
(is (= 0 1))))

0 commit comments

Comments
 (0)