- Understanding Data
Introduction to Data Engineer Basics
Structured, Unstructured, and Semi-Structured
Comparing Data Life Cycles
Section 1: Knowledge Check
- Connecting to Data
Browser User Interface
Import Job vs. Data Link
Section 2: Knowledge Checks
- Going Further with Integration
Going Further with Ingestion
Section 3: Knowledge Check
- Next Steps After Basics
Moving Beyond the Basics
Provide Feedback and Suggested Next Steps
Data Engineer Basics
Data Engineer Basics focuses on just that - the basics, so you can start ingesting data and integrating Datameer within your ecosystem. You will be exposed to Datameer’s ingestion tools, the interface and wizard navigation, and tips on ways to go further with managing your artifacts.
Datameer was architected to meet all the enterprise-class capabilities IT teams demand, covering both the ingestion and integration processes. If you were to stop and think about the current hardware and software in your company’s ecosystem, the list would most likely be a mix of sources of data from smart devices, social media, sensors, machinery, web applications, and a variety of others. By connecting to those sources, you and your team will then be able to analyze the results and combine forces with additional software tools. When performing these tasks and responsibilities of a Data Engineer, we want you to feel confident working with Datameer.
Data Engineer Basics focuses on just that - the basics, so you can start ingesting data and integrating Datameer within your ecosystem. As you move through the training, you will be exposed to Datameer’s ingestion tools, the interface and wizard navigation, and tips on ways to go further with managing your artifacts. Following this course, we invite you to proceed to more advanced integration training within Applied Data Engineering, and then the Datameer Data Engineer Certification.
Version of Datameer
Used to Build this Course
Available Free of Charge
Estimated Time for Completion
3 Sections (each with a short quiz following each section).
Yes, Datameer Data Engineer Certification
Supplement Materials (Y/N)
No, you may complete this course without an instance of Datameer.
Strongly Suggested Prerequisite(s)
At the end of this lesson, you will be able to:
- List examples of machine, human-generated, and organizational data.
- Name three data formats.
- Describe the differences between structured, unstructured, and semi-structured data.
- Memorize data types with the corresponding description of that type, for the following types: Integer, Big Integer, Float, Big Decimal, Date, Boolean, List, and String.
- Differentiate between the traditional ETL model and the modern ELT model.
- List the steps to navigate the browser user interface for:
- the folder structure.
- Creating, opening, and modifying artifacts.
- Reviewing dependencies and artifact details.
- Summarize the main points describing:
- File Uploads
- Import Jobs
- Data Links
- Differentiate between Import Jobs and Data Links.
- Summarize key points used to present SDK overview.
- Explain how intelligent sampling is generated in Datameer.
- Describe scheduling setting options used on artifacts to optimize Datameer and Hadoop resources.
- Explain the steps taken to enable time partitioning within a dataset.
- Recall where to find step-by-step instructions on creating date and time parse patterns.
- Describe steps to reconfigure artifacts in Datameer as saved copies, and saved as new.
- Identify the process and requirements for Exporting Data out of Datameer.
Continue Learning with Datameer:
Datameer eLearning is an online learning library built to teach you big data analytics concepts, simplified via the Datameer platform. Following this course, we invite you to proceed to more advanced integration training including:
- Data Preparation
- Applied Data Engineering
- Datameer Data Engineer Certification
Terms & Conditions
1. Datameer University provides courses, trainings, and regular best practices webinars to get the most out of the Datameer application. We make Datameer University available via a website at 130.105.69 and subdomains of datameer.com and skilljar.com (website), which enables you to use the online courses (services).
3. We can change these terms at any time, whether via our website or by sending you an email. You cannot change these terms except with our prior written consent.
What you get
4. You become a registered user if we have given you a username and password or you have signed up per instructions given by a Datameer representative and you have agreed to follow the website terms and these terms. You need to make sure your username and password are secure and confidential. Make sure you tell us straight away if you think or know someone else has used your password or there has been any other security breach.
5. If you follow these terms, then we grant you a non-exclusive, non-transferable license to access and use the service via the website according to the specifications of the access plan you (or your company) has purchased.
6. You must only use the service and website for your own lawful purposes, in accordance with these terms and any notice sent by us or condition posted on the website.
7. We let you use the service and website to take online courses (courses). If you authorize your employees to take courses, you must make sure that all trainees that access the courses are given, comply with, and accept these terms. You acknowledge that you are solely responsible for your actions while using the website and services and agree to indemnify and defend us from any claims, demands or lawsuits of any kind arising from your breach any of these terms.
What does it cost you?
8. Once you have agreed to purchase Datameer University Enterprise Seats, you need to pay us the annual fee for that plan (and any other taxes or duties) as set out in the order form.
9. Unless otherwise agreed with us in writing, we bill you in advance. You agree that you will pay all charges to your account at any time for the length of your subscription. You can't withhold payment or claim any set-off without getting our written agreement.
10. If you wish to increase your usage of our services, website, or any content provided to over what is stated in your plan, we will bill you for that additional usage, and you agree that you will pay for that additional usage.
11. If at any time you do not make a payment to us when you are supposed to, we can choose to (this doesn’t affect any other rights we may have against you):
a. make you pay, on demand, default interest on any amount you owe us at the greater of 10% per annum calculated on a daily basis, from the date when payment was due until the date when payment is actually made by you or lesser maximum permitted under applicable law. You will also need to pay all expenses and costs (including our legal costs) in connection with us trying to recover any unpaid amount from you; and/or
b. suspend or terminate your use of the service.
What you need to do
12. You need to:
a. make sure you always give us your correct contact and billing details, particularly if these change;
b. follow these terms, the website terms and other agreements you have with us;
c. make sure that you comply with all laws, regulations, ordinances and other rules that relate to your use of the website, and the service.
13. You remain solely responsible for complying with all applicable laws with regard to the website, and the service. For example, it is your responsibility to check that storage of and access to your data via the website will comply with laws applicable to you (including any laws requiring you to retain records).
What we don’t let you do
14. You can’t:
a. assign or transfer any rights you have under these terms to any other person without getting our written agreement;
b. do anything that would damage or disrupt our website or a service or anyone else’s use of our website or a service;
c. use our website or a service, including, without limitation, any communication tools available through the website such as Skype, any forum, chat room or message centre:
i. to upload material or data in violation of any law (including to breach copyright or other intellectual property rights held by us or anyone else which you do not have the right to use);
ii. to send unwelcome communications of any kind to anyone (e.g. spam or chain letters);
iii. to abuse, defame, threaten, stalk or harass anyone;
iv. to publish, post, upload or distribute unsuitable, offensive, obscene or discriminatory information of any kind;
v. for running any network scanning software, spiders, spyware software, robots, open relay software or similar software;
vi. to upload anything or otherwise introduce any viruses, worms, trojan horses, time bombs or bots or any other damaging items which could interfere with our, or anyone else’s, network or computer system;
vii. for using any software or device which may hinder the services (like mail bombs, war dialing, pinging, etc.);
viii. to attempt to gain unauthorized access to any services other than those to which you have been given express permission to access; or
ix. for using a false identity to try to trick anyone for any reason.
a. We will not provide any part-month refund for any time not used in the month you cancel the service. We may terminate your access to the website with 30 days' notice sent to you via our website or by sending you an e-mail.
b. For annual customers, your contract automatically renews for 12-month periods on the anniversary of the original contract or upgrade, unless either you or we send a notice of non-renewal at least thirty days prior to the annual anniversary date.
c. We do not provide any refunds on pre-paid fees.
16. Uncured Breach. Either you or we may terminate these terms if the other one of us breaches these terms and does not cure such breach within thirty days of receiving a termination for breach notice. In such circumstances you acknowledge that we may delete your data after termination.
17. When you make any communication on the website, you represent that you are permitted to make such communication. We are under no obligation to ensure that the communications on the website are legitimate or that they are related only to the use of the services. You agree to indemnify and defend us from any claims, demands or lawsuits of any kind arising from your communications on the website.
18. As with any other web-based forum, you must exercise caution when using the communication tools available on the website. However, while we are not obligated to, we have the right to remove any communication at any time.
19. WE DON’T GIVE YOU ANY WARRANTY OR UNDERTAKING ABOUT THE SERVICES OR THE WEBSITE AS THEY ARE PROVIDED “AS IS”. TO AVOID DOUBT, ALL IMPLIED CONDITIONS OR WARRANTIES ARE EXCLUDED AS MUCH AS IS PERMITTED BY LAW, INCLUDING (WITHOUT LIMITATION) WARRANTIES OF MERCHANTABILITY, FITNESS FOR PURPOSE, TITLE AND NON-INFRINGEMENT.
20. We will try to give you access to the website all the time but we do not make any promises or provide you with a warranty that the website or the services will be without any faults or interruptions; be secure or private or that it won’t have any viruses or other similar nasty things.
21. Whilst we intend that the services should be available 24 hours a day, seven days a week, it is possible that on occasions the website or services may be unavailable to permit maintenance or other development activity to take place. If for any reason we have to interrupt the services for longer periods than we would normally expect, we will use reasonable endeavors to publish in advance details of such activity on the website.
22. Information on our website may change regularly. We will try to keep our website up to date and correct but again we do not make any promises or guarantees about the accuracy of the information on our website.
23. We do not warrant that the services will meet your requirements or that they will be suitable for any particular purpose. It is your sole responsibility to determine that the services meet the needs of your business or otherwise and are suitable for the purposes for which they are used.
24. We also aren’t legally responsible for:
a. any corruption or loss of data or other content which you or anyone else may experience after using our website, or any problems you may have when you view or navigate our website;
b. devices or equipment that we do not own or have not given you;
c. if you do not follow our instructions or these terms or the website terms;
d. any actions or non-actions or other people which disrupts access to our website including the trainees;
e. the content of ads appearing on our website (including links to advertisers' own websites) as the advertisers are responsible for the ads (we don't endorse the advertisers' products);
f. the content of other people’s websites even if a link to their website is included on the website (we just include the links for convenience to you).
25. You warrant that if you are accessing and using the services for the purposes of a business then, to the maximum extent permitted by law, any statutory consumer guarantees or legislation intended to protect non-business consumers in any jurisdiction do not apply to the supply of the services, the website or these terms.
26. You agree not to do or omit to do anything which gives rise to any liability on your part or our part under any legislation or omit to do anything which would avoid that liability. This includes not making any representation or giving any guarantee, warranty or other undertaking to anyone including your trainees in relation to the services unless that representation, guarantee, warranty or undertaking is supplied by us.
Limitation of liability
27. TO THE MAXIMUM EXTENT PERMITTED BY LAW, WE (THIS INCLUDES OUR OFFICERS, STAFF AND AGENTS) ARE NOT LIABLE (UNDER ANY LEGAL THEORY) FOR DAMAGES OR OTHER AMOUNTS THAT EXCEED THE SUM OF THE FEES PAID BY YOU FOR THE PREVIOUS 6 MONTHS, AND WE ARE NOT LIABLE FOR CONSEQUENTIAL, INCIDENTAL, INDIRECT, SPECIAL, EXEMPLARY, OR PUNITIVE DAMAGES, OR LOSS OF PROFITS, LOSS OF DATA OR REVENUE, LOSS OF USE, LOST BUSINESS OR MISSED OPPORTUNITIES, WASTED EXPENDITURE OR SAVINGS WHICH YOU MIGHT HAVE HAD, DENIAL OF SERVICE OR ACCESS TO OUR WEBSITE, OCCURRING DIRECTLY OR INDIRECTLY FROM THE USE OR ABILITY OR INABILITY TO USE, OR RELIANCE ON, OUR WEBSITE, OR THE SERVICE. THE FOREGOING LIMITATONS OF LIABILITY AND DISCLAIMERS OF DAMAGES WILL NOT APPLY TO OUR LIABILITY FOR BODILY INJURTY, DEATH, OR PHYSICAL DAMAGE TO YOUR TANGIBLE PERSONAL PROPERTY ATTRIBUTABLE TO OUR MALICIOUS MISCONDUCT OR GROSS NEGLIGENCE. YOU SHALL INDEMNIFY US AGAINST ALL CLAIMS, COSTS (INCLUDING ALL OUR ATTORNEY FEES), EXPENSES, DEMANDS OR LIABILITY, DAMAGES AND LOSSES WITHER DIRECT, INDIRECT, CONSEQUENTIAL, OR OTHERWISE, AND WHETHER ARISING IN CONTRACT, TORT (INCLUDING IN EACH CASE NEGLIGENCE), OR EQUITY OR OTHERWISE ARISING FROM YOUR, OR ANY OF YOUR TRAINEE’S, BREACH OF ANY OF THESE TERMS OR ANY OBLIGATIONS THAT YOU OR THEY MAY HAVE TO US.
28. If you are not satisfied with the services, then your sole and exclusive remedy is to terminate these terms, in accordance with the Termination section.
29. We own or have the right to use the copyright, trademarks, design rights, moral rights, software and other similar rights (intellectual property rights) with respect to the website, the service, or the content licensed to you from us, or content of a third party provider supplied to you by us, (content) (our IP). You can’t reproduce any of the content on our website, nor make derivative works of our IP, without getting our express permission in writing. You do not have any rights to our IP except granted under these terms or any other agreement you have with us.
30. You are not allowed to, and you can’t let anyone else, copy, alter, distribute, display, license, modify or reproduce, reverse assemble, reverse compile (whether digitally, electronically, by linking, or in hard copy or by any means whatsoever) or use any of our IP without getting our permission first in writing, unless in order to use our website the way you are supposed to use it you need to copy, reproduce or use the IP.
31. You need to tell us straight away if you think someone has infringed our IP, or someone has threatened to or you suspect someone might. You also need to tell us as soon as you become aware that someone claims any use of our website infringes the rights of anyone else. If this happens you will need to do everything we ask you to reasonably do (we will pay for your costs) to help us pursue or defend any associated proceedings.
32. You own, or warrant that you are authorized to use, the intellectual property rights in any data you submit to the website. However, your access to the data depends on you paying your fees when due. You grant us a license to use, copy, transmit, store, and back-up your information and data for the purposes of enabling you and your trainees to access and use the services and for any other purpose related to provision of services to you.
33. You must maintain copies of all data inputted into the services. We adhere to best practice policies and procedures to prevent data loss, including a daily system data back-up regime, but do not make any guarantees that there will be no loss of data. It is also up to you to remove all data prior to you terminating these terms with us.
34. We can use any information we have about you relating to your credit worthiness and give that information to any other person for credit assessment and debt collection purposes.
35. Your data is yours and is confidential until such time as it becomes public (through no fault of ours). We will not use it, disclose it or even look at it unless we have to for security or operational purposes (e.g. if it seems to be causing technical problems), or it breaches these terms or if we are required by law to do so (for example where we are legally required to disclose it).
36. We might however collect and keep information about you that you have given us or that we have gained from another source. You agree that we can use your information for dealing internally with requests, complaints and other customer activities, market and product research and to be able to give you promotional material on our other services and special offers.
37. We will collect your user information about your visits to our website to measure the amount of visitors to different parts of the site. We may gather more information if we are worried about security issues for example. If we have to by law then we might give any information or data to the authorities.
38. We keep all of the personal information and your data that we collect confidential to the same extent as our confidential information, and in compliance with all applicable laws (this includes your contact details). However you will no doubt understand that we cannot guarantee the security of any transmission.
40. You must not disclose or make our confidential information available to any person, or use the same for your own benefit, other than as contemplated by these terms.
41. You can contact us by writing to or to any other email address notified by email to you by us, with copies of any legal notices sent, Datameer Inc., 1550 Bryant St., Ste 490, San Francisco, Ca 94103, Attention: Legal Department. If we need to contact you, we will email the email address you gave us when you set up access to the services.
No Rights of Third Parties
42. A person who is not a party to these terms, including a trainee, has no right to benefit under, nor to enforce any, term of these terms.
43. These terms and our relationship are governed by law of the State of California without regard to its conflict of laws provisions. The parties expressly exclude the application of the United Nations Convention on Contracts for the International Sale of Goods. In any litigation, the prevailing party shall be reimbursed reasonable court costs and attorneys’ fees.
By ticking the box and accepting, you confirm your agreement to these terms on behalf of the organization you are registering and you confirm that you personally are authorized to accept this agreement on that organization’s behalf. This is a binding legal agreement so don't accept unless you have the organization's authority to do so.
These Terms & Conditions were last updated at Apirl 4 , 2017.