Big Data Training in Matosinhos

This instructor-led, live training in Matosinhos (online or onsite) is aimed at intermediate-level data scientists and engineers who wish to use Google Colab and Apache Spark for big data processing and analytics. By the end of this training, participants will be able to:

• Set up a big data environment using Google Colab and Spark.
• Process and analyze large datasets efficiently with Apache Spark.
• Visualize big data in a collaborative environment.
• Integrate Apache Spark with cloud-based tools.

Participants who complete this instructor-led, live training in Matosinhos will gain a practical, real-world understanding of Big Data and its related technologies, methodologies and tools. Participants will have the opportunity to put this knowledge into practice through hands-on exercises. Group interaction and instructor feedback make up an important component of the class. The course starts with an introduction to elemental concepts of Big Data, then progresses into the programming languages and methodologies used to perform Data Analysis. Finally, we discuss the tools and infrastructure that enable Big Data storage, Distributed Processing, and Scalability.

This instructor-led, live training in Matosinhos (online or onsite) is aimed at beginner to intermediate-level data analysts and data scientists who wish to use Weka to perform data mining tasks. By the end of this training, participants will be able to:

• Install and configure Weka.
• Understand the Weka environment and workbench.
• Perform data mining tasks using Weka.

This instructor-led, live training in Matosinhos (online or onsite) is aimed at intermediate-level big data professionals who wish to utilize Apache Kylin for building real-time data warehouses and performing multidimensional analysis on large-scale datasets.By the end of this training, participants will be able to:

• Set up and configure Apache Kylin with real-time streaming data sources.
• Design and build OLAP cubes for both batch and streaming data.
• Perform complex queries with sub-second latency using Kylin's SQL interface.
• Integrate Kylin with BI tools for interactive data visualization.
• Optimize performance and manage resources effectively in Kylin.

This instructor-led, live training in Matosinhos (online or onsite) is aimed at beginner-level to intermediate-level system administrators who wish to deploy, maintain, and optimize Spark clusters. By the end of this training, participants will be able to:

• Install and configure Apache Spark in various environments.
• Manage cluster resources and monitor Spark applications.
• Optimize the performance of Spark clusters.
• Implement security measures and ensure high availability.
• Debug and troubleshoot common Spark issues.

Participants who complete this instructor-led, live training in Matosinhos will gain a practical, real-world understanding of Big Data and its related technologies, methodologies and tools. Participants will have the opportunity to put this knowledge into practice through hands-on exercises. Group interaction and instructor feedback make up an important component of the class. The course starts with an introduction to elemental concepts of Big Data, then progresses into the programming languages and methodologies used to perform Data Analysis. Finally, we discuss the tools and infrastructure that enable Big Data storage, Distributed Processing, and Scalability.

In this instructor-led, live training in Matosinhos, participants will learn how to use Python and Spark together to analyze big data as they work on hands-on exercises. By the end of this training, participants will be able to:

• Learn how to use Spark with Python to analyze Big Data.
• Work on exercises that mimic real world cases.
• Use different tools and techniques for big data analysis using PySpark.

This course is intended for developers and data scientists who want to understand and implement artificial intelligence in their applications. Special focus is placed on data analytics, distributed AI, and natural language processing.

Apache ActiveMQ is an open source message broker written in Java.

Advances in technologies and the increasing amount of information are transforming how business is conducted in many industries, including government. Government data generation and digital archiving rates are on the rise due to the rapid growth of mobile devices and applications, smart sensors and devices, cloud computing solutions, and citizen-facing portals. As digital information expands and becomes more complex, information management, processing, storage, security, and disposition become more complex as well. New capture, search, discovery, and analysis tools are helping organizations gain insights from their unstructured data. The government market is at a tipping point, realizing that information is a strategic asset, and government needs to protect, leverage, and analyze both structured and unstructured information to better serve and meet mission requirements. As government leaders strive to evolve data-driven organizations to successfully accomplish mission, they are laying the groundwork to correlate dependencies across events, people, processes, and information. High-value government solutions will be created from a mashup of the most disruptive technologies:

• Mobile devices and applications
• Cloud services
• Social business technologies and networking
• Big Data and analytics

IDC predicts that by 2020, the IT industry will reach $5 trillion, approximately $1.7 trillion larger than today, and that 80% of the industry's growth will be driven by these 3rd Platform technologies. In the long term, these technologies will be key tools for dealing with the complexity of increased digital information. Big Data is one of the intelligent industry solutions and allows government to make better decisions by taking action based on patterns revealed by analyzing large volumes of data — related and unrelated, structured and unstructured. But accomplishing these feats takes far more than simply accumulating massive quantities of data.“Making sense of thesevolumes of Big Datarequires cutting-edge tools and technologies that can analyze and extract useful knowledge from vast and diverse streams of information,” Tom Kalil and Fen Zhao of the White House Office of Science and Technology Policy wrote in a post on the OSTP Blog. The White House took a step toward helping agencies find these technologies when it established the National Big Data Research and Development Initiative in 2012. The initiative included more than $200 million to make the most of the explosion of Big Data and the tools needed to analyze it. The challenges that Big Data poses are nearly as daunting as its promise is encouraging. Storing data efficiently is one of these challenges. As always, budgets are tight, so agencies must minimize the per-megabyte price of storage and keep the data within easy access so that users can get it when they want it and how they need it. Backing up massive quantities of data heightens the challenge. Analyzing the data effectively is another major challenge. Many agencies employ commercial tools that enable them to sift through the mountains of data, spotting trends that can help them operate more efficiently. (A recent study by MeriTalk found that federal IT executives think Big Data could help agencies save more than $500 billion while also fulfilling mission objectives.). Custom-developed Big Data tools also are allowing agencies to address the need to analyze their data. For example, the Oak Ridge National Laboratory’s Computational Data Analytics Group has made its Piranha data analytics system available to other agencies. The system has helped medical researchers find a link that can alert doctors to aortic aneurysms before they strike. It’s also used for more mundane tasks, such as sifting through résumés to connect job candidates with hiring managers.

This classroom based training session will explore Big Data. Delegates will have computer based examples and case study exercises to undertake with relevant big data tools

Day 1 - provides a high-level overview of essential Big Data topic areas. The module is divided into a series of sections, each of which is accompanied by a hands-on exercise. Day 2 - explores a range of topics that relate analysis practices and tools for Big Data environments. It does not get into implementation or programming details, but instead keeps coverage at a conceptual level, focusing on topics that enable participants to develop a comprehensive understanding of the common analysis functions and features offered by Big Data solutions. Day 3 - provides an overview of the fundamental and essential topic areas relating to Big Data solution platform architecture. It covers Big Data mechanisms required for the development of a Big Data solution platform and architectural options for assembling a data processing platform. Common scenarios are also presented to provide a basic understanding of how a Big Data solution platform is generally used.  Day 4 - builds upon Day 3 by exploring advanced topics relatng to Big Data solution platform architecture. In particular, different architectural layers that make up the Big Data solution platform are introduced and discussed, including data sources, data ingress, data storage, data processing and security.  Day 5 - covers a number of exercises and problems designed to test the delegates ability to apply knowledge of topics covered Day 3 and 4. 

In this instructor-led, live training in Matosinhos, participants will learn the mindset with which to approach Big Data technologies, assess their impact on existing processes and policies, and implement these technologies for the purpose of identifying criminal activity and preventing crime. Case studies from law enforcement organizations around the world will be examined to gain insights on their adoption approaches, challenges and results.By the end of this training, participants will be able to:

• Combine Big Data technology with traditional data gathering processes to piece together a story during an investigation.
• Implement industrial big data storage and processing solutions for data analysis.
• Prepare a proposal for the adoption of the most adequate tools and processes for enabling a data-driven approach to criminal investigation.

Big Data is a term that refers to solutions destined for storing and processing large data sets. Developed by Google initially, these Big Data solutions have evolved and inspired other similar projects, many of which are available as open-source. R is a popular programming language in the financial industry.

When traditional storage technologies don't handle the amount of data you need to store there are hundereds of alternatives. This course try to guide the participants what are alternatives for storing and analyzing Big Data and what are theirs pros and cons. This course is mostly focused on discussion and presentation of solutions, though hands-on exercises are available on demand.

The course is part of the Data Scientist skill set (Domain: Data and Technology).

This instructor-led, live training (online or onsite) is aimed at engineers who wish to use Confluent (a distribution of Kafka) to build and manage a real-time data processing platform for their applications. By the end of this training, participants will be able to:

• Install and configure Confluent Platform.
• Use Confluent's management tools and services to run Kafka more easily.
• Store and process incoming stream data.
• Optimize and manage Kafka clusters.
• Secure data streams.

Format of the Course

• Interactive lecture and discussion.
• Lots of exercises and practice.
• Hands-on implementation in a live-lab environment.

Course Customization Options

• This course is based on the open source version of Confluent: Confluent Open Source.
• To request a customized training for this course, please contact us to arrange.

Audience If you try to make sense out of the data you have access to or want to analyse unstructured data available on the net (like Twitter, Linked in, etc...) this course is for you. It is mostly aimed at decision makers and people who need to choose what data is worth collecting and what is worth analyzing. It is not aimed at people configuring the solution, those people will benefit from the big picture though. Delivery Mode During the course delegates will be presented with working examples of mostly open source technologies. Short lectures will be followed by presentation and simple exercises by the participants Content and Software used All software used is updated each time the course is run, so we check the newest versions possible. It covers the process from obtaining, formatting, processing and analysing the data, to explain how to automate decision making process with machine learning.

Course can be provided with any tools, including free open-source data mining software and applications

R is an open-source free programming language for statistical computing, data analysis, and graphics. R is used by a growing number of managers and data analysts inside corporations and academia. R has a wide variety of packages for data mining.

Big data is data sets that are so voluminous and complex that traditional data processing application software are inadequate to deal with them. Big data challenges include capturing data, data storage, data analysis, search, sharing, transfer, visualization, querying, updating and information privacy.

This instructor-led, live training in Matosinhos (online or onsite) is aimed at beginner-level professionals who wish to gain a mastery of the fundamentals of R and how to work with data.By the end of this training, participants will be able to:

• Understand the R programming environment and RStudio interface.
• Import, manipulate, and explore datasets using R commands and packages.
• Perform basic statistical analysis and data summarization.
• Generate visualizations using both base R and ggplot2.
• Manage workspaces, scripts, and packages effectively.

Apache Hadoop is the most popular framework for processing Big Data on clusters of servers. In this three (optionally, four) days course, attendees will learn about the business benefits and use cases for Hadoop and its ecosystem, how to plan cluster deployment and growth, how to install, maintain, monitor, troubleshoot and optimize Hadoop. They will also practice cluster bulk data load, get familiar with various Hadoop distributions, and practice installing and managing Hadoop ecosystem tools. The course finishes off with discussion of securing cluster with Kerberos. “…The materials were very well prepared and covered thoroughly. The Lab was very helpful and well organized”
— Andrew Nguyen, Principal Integration DW Engineer, Microsoft Online Advertising Audience Hadoop administrators Format Lectures and hands-on labs, approximate balance 60% lectures, 40% labs.

Apache Hadoop is one of the most popular frameworks for processing Big Data on clusters of servers. This course delves into data management in HDFS, advanced Pig, Hive, and HBase.  These advanced programming techniques will be beneficial to experienced Hadoop developers. Audience: developers Duration: three days Format: lectures (50%) and hands-on labs (50%).

This instructor-led, live training in Matosinhos (online or onsite) is aimed at technical persons who wish to learn how to implement a machine learning strategy while maximizing the use of big data. By the end of this training, participants will:

• Understand the evolution and trends for machine learning.
• Know how machine learning is being used across different industries.
• Become familiar with the tools, skills and services available to implement machine learning within an organization.
• Understand how machine learning can be used to enhance data mining and analysis.
• Learn what a data middle backend is, and how it is being used by businesses.
• Understand the role that big data and intelligent applications are playing across industries.

Apache Spark's learning curve is slowly increasing at the begining, it needs a lot of effort to get the first return. This course aims to jump through the first tough part. After taking this course the participants will understand the basics of Apache Spark , they will clearly differentiate RDD from DataFrame, they will learn Python and Scala API, they will understand executors and tasks, etc.  Also following the best practices, this course strongly focuses on cloud deployment, Databricks and AWS. The students will also understand the differences between AWS EMR and AWS Glue, one of the lastest Spark service of AWS.   AUDIENCE: Data Engineer, DevOps, Data Scientist

This instructor-led, live training in Matosinhos (online or onsite) is aimed at data scientists and developers who wish to use Spark NLP, built on top of Apache Spark, to develop, implement, and scale natural language text processing models and pipelines. By the end of this training, participants will be able to:

• Set up the necessary development environment to start building NLP pipelines with Spark NLP.
• Understand the features, architecture, and benefits of using Spark NLP.
• Use the pre-trained models available in Spark NLP to implement text processing.
• Learn how to build, train, and scale Spark NLP models for production-grade projects.
• Apply classification, inference, and sentiment analysis on real-world use cases (clinical data, customer behavior insights, etc.).

Spark SQL is Apache Spark's module for working with structured and unstructured data. Spark SQL provides information about the structure of the data as well as the computation being performed. This information can be used to perform optimizations. Two common uses for Spark SQL are:
- to execute SQL queries.
- to read data from an existing Hive installation. In this instructor-led, live training (onsite or remote), participants will learn how to analyze various types of data sets using Spark SQL. By the end of this training, participants will be able to:

• Install and configure Spark SQL.
• Perform data analysis using Spark SQL.
• Query data sets in different formats.
• Visualize data and query results.

Format of the Course

• Interactive lecture and discussion.
• Lots of exercises and practice.
• Hands-on implementation in a live-lab environment.

Course Customization Options

• To request a customized training for this course, please contact us to arrange.

MLlib is Spark’s machine learning (ML) library. Its goal is to make practical machine learning scalable and easy. It consists of common learning algorithms and utilities, including classification, regression, clustering, collaborative filtering, dimensionality reduction, as well as lower-level optimization primitives and higher-level pipeline APIs. It divides into two packages:

• spark.mllib contains the original API built on top of RDDs.
• spark.ml provides higher-level API built on top of DataFrames for constructing ML pipelines.

Audience This course is directed at engineers and developers seeking to utilize a built in Machine Library for Apache Spark

This instructor-led, live training in Matosinhos (online or onsite) is aimed at technical persons who wish to deploy Talend Open Studio for Big Data to simplifying the process of reading and crunching through Big Data. By the end of this training, participants will be able to:

• Install and configure Talend Open Studio for Big Data.
• Connect with Big Data systems such as Cloudera, HortonWorks, MapR, Amazon EMR and Apache.
• Understand and set up Open Studio's big data components and connectors.
• Configure parameters to automatically generate MapReduce code.
• Use Open Studio's drag-and-drop interface to run Hadoop jobs.
• Prototype big data pipelines.
• Automate big data integration projects.