This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
begin transaction; | |
drop table if exists numbers; | |
create temp table numbers as ( | |
with recursive | |
counts (number) as ( | |
select 1 as number | |
union all | |
select number + 1 | |
from counts m |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
""" | |
Universidade de São Paulo | |
Escola Politécnica | |
Especializaçäo em Engenharia de Dados & Big Data | |
Análise e Mineração de Dados - eEDB-001 | |
Atividade 04 | |
Desenvolvimento de função que realiza validação cruzada no treinamento de | |
modelo de regressão logística. | |
""" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
""" | |
Main ETL script | |
Formato table final | |
Nome do Banco | |
CNPJ | |
Classificação do Banco | |
Quantidade de Clientes do Bancos | |
Índice de reclamações | |
Quantidade de reclamações |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Copiar o conteúdo em arquivo com nome `docker-compose.yml` | |
# Em seguida, no terminal aonde está localizado o arquivo rodar: docker-compose up | |
services: | |
db: | |
image: mysql | |
environment: | |
MYSQL_ROOT_PASSWORD: root | |
MYSQL_DATABASE: mydb | |
MYSQL_USER: user | |
MYSQL_PASSWORD: pass |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
""" | |
Connect Amazon S3 file to PowerBI | |
""" | |
# Make sure these packages are installed in your local | |
# because the script will run in your local default Python | |
# otherwise: `pip install boto3` etc. | |
import os | |
import io | |
import boto3 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# This .gitconfig file will definitely help to increase your productivity | |
# It can be accessed through the Visual Studio Code, whithin your user folder, open the CMD and type: "code .gitconfig". | |
# You need to have the VSCode mapped in your Environment Variables | |
[user] | |
name = ____ | |
email = ____@____ | |
[color] | |
added = green |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
{ | |
"workbench.colorTheme": "Dracula", | |
"workbench.iconTheme": "material-icon-theme", | |
"editor.tabSize": 2, | |
"editor.fontSize": 14, | |
"editor.lineHeight": 26, | |
"editor.fontFamily": "Fira Code", | |
"editor.fontLigatures": true, | |
"editor.semanticHighlighting.enabled": false, |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
---================================================================================================-- | |
-- SIMEX - Ranking | |
-- Rondônia | |
-- Agosto 2019 - Julho 2020 | |
-- | |
-- Author: Tomás Carvalho | |
-- Data: Abr/2021 | |
---================================================================================================-- | |
--== Autex agrupada por Número de Série Data: > 2019-08-01 ==-- | |
CREATE MATERIALIZED VIEW ro.mv_autorizao_exploracao_dof_ibama_group AS |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
---================================================================================================-- | |
-- SIMEX | |
-- Rondônia | |
-- Agosto 2019 - Julho 2020 | |
-- | |
-- Author: Tomás Carvalho | |
-- Data: Abr/2021 | |
---================================================================================================-- | |
--== Autex agrupada por Número de Série Data: > 2019-08-01 ==-- | |
--DROP MATERIALIZED VIEW ro.mv_autorizao_exploracao_dof_ibama_group CASCADE; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
### Testado utilizando python 3.7 | |
## requirements: | |
## pip intall psycopg2 sqlalchemy pandas | |
import psycopg2 | |
from sqlalchemy import create_engine | |
import pandas as pd | |
from datetime import date |