Project BigData/IA/WEB
Project realized in the 1st year engineering school,
involving the design and development of an application to study
the tree heritage of Saint-Quentin (Aisne). The project took place
over 3 weeks, each of which was dedicated to a subject
(BigData/ AI/ WEB).
BigData part
The BigData part has 5 parts which were expected:
1: Data description and exploration.
2: Visualize data graphically.
3: View data on a map.
4: Study the correlation of variables.
5: Predicting the estimated age of a tree.
Feature 1
Data description and exploration :
Dataset description (describe what each variable in the initial csv file corresponds to)
Univariate and bivariate descriptive statistics
Data cleaning :
- missing values, outliers values
- Duplicates.
Feature 2
Visualize data graphically.t
Here are a few examples of graphics that were useful for the project:
Feature 3
View data on a map.
Map of Saint-Quentin and surroundings
Tree position :
-Green Point: trees in place
-Grey Point: felled trees
-Red Point: remarkable trees (regardless of whether they are in place or not)
Distribution of trees by neighborhood :
- The color is blue when the neighborhood has many trees.
- The color becomes redder as the number of trees decreases.
Feature 4
Study the correlation of variables.
There was a significant correlation (0.77 and 0.80) between estimated tree age and trunk diameter,
as well as between estimated tree age and estimation accuracy.
Feature 5
Predicting the estimated age of a tree.
In this part we want to predict the tree age variable, to do this, we'll carry out a regression study of the tree age variable.
We then want to know which trees to cut down, and we'll use a logistic regression study to find out. Our linear regression model
has a low residual error, which proves that the model can estimate tree age well. tree age.
For more information, here's the Big Data report (French)
Click here
IA part
3 Expected customer requirements
Customer requirement 1: View on a map
Customer requirement 2: Age prediction model
Customer requirement 3: Storm warning system
Customer requirement 1: View on a map
Unsupervised learning
This part involved the use of several clustering algorithms
to separate trees into groups based on their size, and
determine the appropriate number of clusters.
Metrics for Unsupervised Learning
Evaluation of the quality of the clustering performed in the previous step using metrics :
- Silhoutete score: Measures how well a data point is assigned to its own cluster cluster compared to other clusters
- The Calinski-Harabasz Index: measures dispersion between clusters in relation to dispersion within clusters.
- Davies-Bouldin Index: Measures the average of the ratios between intra-cluster dispersion and inter-cluster separation
View on a map
Click here
A script has been created to allow a user to choose the number of clusters of his choice on the map.
Here's the map when the user chooses 3 clusters
Customer requirement 2: Age prediction model
In Requirement 2, we target the estimated age of the trees, unlike in Customer Requirement 1,
where we use Supervised Learning. Metrics will be used to evaluate the classification models.
We've also created a script that takes a dataFrame(.JSON) as input and returns the age prediction in
.JSON format.
Customer requirement 3: Storm warning system
In Requirement 3, we target the state of the trees, using Supervised Learning for classification,
which will be evaluated using metrics. A script has been created to select the classification method
to be used, which returns a map of trees at risk of being uprooted.
Here's an example of the map when you choose the “Random Forest” method
Click here
For more information, here's the report on the AI report (French)
Click ici
WEB part
Clicking on the button will take you to the site mockup with only a small portion of the database, to avoid script latency.
Click here
