When MAVEN_OPTS=-javaagent:log_agent-0.0.2-SNAPSHOT.jar is set, the next step is to save it in the file named .mavenrc. Then just launch mvn command in the folder with pom.xml file and all dependencies will be included by Maven what is convenient.
1 Comment
First line tells the system what program should run the file.
#!/bin/bash for i in {1..5} do echo "Welcome $i times" done The next step is to make a file executable with the next command. chmod +x filename *.sh Finally, it can be started with the next command. ./filename Look that not sh in front of file name, but only dot and slash combination of symbols. http://tldp.org/HOWTO/Bash-Prog-Intro-HOWTO.html This assignment was a final one for an introduction course on Coursera. It was well-structured, but I spent a lot of time because tried to parse a string instead of an object that is returned from a data set and used incorrect indices. When this issue was solved, other part was quite short. Thanks to colleagues.
show_views_file = sc.textFile("inputjoin/join2_gennum?.txt") def split_show_views(line): keyValue = line.split(',') show = keyValue[0] views = int(keyValue[1]) return (show, views) show_views = show_views_file.map(split_show_views) show_channel_file = sc.textFile("inputjoin/join2_genchan?.txt") def split_show_channel(line): keyValue = line.split(',') show = keyValue[0] channel = keyValue[1] return (show, channel) show_channel = show_channel_file.map(split_show_channel) joined_dataset = show_channel.join(show_views) def extract_channel_views(show_views_channel): aline = show_views_channel[1] views = int(aline[1]) channel = aline[0] return (channel, views) channel_views = joined_dataset.map(extract_channel_views) def sum_function(a, b): sum_value = a + b return sum_value channel_views.reduceByKey(sum_function).collect() . def split_fileA(line):
# split the input line in word and count on the comma keyValue = line.split(',') # turn the count to an integer word = keyValue[0] count = int(keyValue[1]) return (word, count) def split_fileB(line): # split the input line into word, date and count_string keyValue = line.split(',') bigKey = keyValue[0].split(' ') word = bigKey[1] date = bigKey[0] count_string = keyValue[1] return (word, date + " " + count_string) It should be placed to Spark console that can be initiated with command PYSPARK_DRIVER_PYTHON=ipython pyspark . Moreover, one may need to install ipython on machine. The first week of a quartile and very interesting. I tried to work on tasks from Coursera, find information about homework related to internet of Things course and participate in projects in the university.
In the afternoon on Friday I had a first instruction class as a student assistant. It was interesting and exciting. The whole day before I studied the same lectures as those given to students. There are a lot of them, and I just performed a small portion of them. It was useful. However, the main question during the instruction was about installation of local web server. Many people had different issues with it, but they performed well and mainly could install it during the class. Yesterday we had a nice walk with friends. We would like to wish the best to Jonas who is moving to the capital of Belgium where he will be working on research project. It is a great opportunity, and he is a smart folk. We sure that he should do it decently. After that, I returned home and continued to work on assignments. First of all it is necessary to install MySQL server. It is possible to do by following this guide http://dev.mysql.com/doc/mysql-apt-repo-quick-guide/en/ . A good piece of advice to save *.deb file and run it with CLI manually, because GUI didn't respond for a long time while I tried to install it with it. Then, check whether the program works
sudo service mysql status . Information about current status should be presented. Then, important to install RMySQL package in R. It depends on C files and they can be installed with (thanks to http://www.patlaf.com/install-rmysql-on-ubuntu/) the next command sudo apt-get install libmysqlclient-dev . If everything is working as it should, just run install.packages("RMySQL") in R and it should work. Some time ago, I have decided to purchase verified certificates on Coursera. It is a nice website, and to spend some money on a good thing like certificates and to support a website was a conscious decision.
Money were paid, then it is a time to work hard to prove that they were spent in a reasonable way. I started with Getting and Cleaning data. This course seems to be more difficult than a previous one, but useful. I had some difficulties with installation of R packages, and solution was to change owner for a directory and allow to write files there. In addition, some packages required Java and do not work without it. It useful to run command sudo R CMD javareconf . When it was typed in CLI, everything started to work as it should. It was an advice from Stackoverflow site. It was so easy and practical advice that worth to save. It is assumed that mvn archetype:generate command has been executed. An example about the usage of this command can be found in the previous post.
Artifact in this environment is a project. These words are used as synonyms on the official website. In command line type mvn package Each project has a build lifecycle. It includes many phases that are performed during it. The list of phases for a default lifecycle is shown next.
When run only one phase like deploy, it forces all previous phases to be performed too. For example, we can validate the project. mvn validate There are still some issues with packages, because when I try to run created package java returns an error "no main manifest attribute, in my-app-1.0-SNAPSHOT.jar" and don't run mentioned package. Possible solution is presented here http://stackoverflow.com/questions/9689793/cant-execute-jar-file-no-main-manifest-attribute . To make a jar executable you need to jar a file called META-INF/MANIFEST.MF the file itself should have (at least) this one liner: Main-Class: com.mypackage.MyClass , where com.mypackage.MyClass is the class holding the public static void main(String[] args) entry point. The main idea is to make a file executable. A nice instruction how to do that is presented here. http://stackoverflow.com/questions/574594/how-can-i-create-an-executable-jar-with-dependencies-using-maven?lq=1 When mvn package was executed, it is time to check whether jar works. From the directory with pom.xml run the command
It should display Hello, world! message to the line. Structure of a default project is presented at this https://maven.apache.org/guides/introduction/introduction-to-the-standard-directory-layout.html page.
Folder should contain src subdirectory and three *.txt files.
At this moment, I am not sure what are filters and how they should be used in a project. This is a structure presented in a tutorial at maven official website. my-app |-- pom.xml `-- src |-- main | `-- java | `-- com | `-- mycompany | `-- app | `-- App.java `-- test `-- java `-- com `-- mycompany `-- app `-- AppTest.java The src/main/java directory contains the source code, src/test/java contains test and pom.xml is a Project Object Model for this project. This week is rather different from a usual one. Since Tuesday I am almost free in choice what activities to choose.
Yesterday, I spent the majority of my time on BSR project. It was very useful, because it allows me to learn Java while trying to answer a research questions. The first meeting was at this morning and I have quite nice experience for it. We need to collaborate more with my partner. A very popular educational platform Coursera allows you to follow courses from the top universities all over the world. They are not easy, but really make you to start thinking in the right direction. I'm trying to complete a course about Algorithms from Princeton, and it is really challenging. |