Archive for the ‘Statistik & Zahlen’ Category.

Approximating Pi with Python

One of the many approximations of Pi is the Gregory-Leibniz series (image source Wikipedia):

Leibnis Series for Pi

Here is an implementation in Python:

# -*- coding: utf-8 -*-
"""
Created on Sat Mar 25 06:52:11 2017
@author: Uwe Ziegenhagen
"""
 
sum = 0
factor = -1
 
for i in range(1, 640000, 2):
        sum = sum + factor * 1/i
        factor *= -1
        # print(sum)
 
print(-4*sum)

Uwe

Uwe Ziegenhagen has been working with LaTeX for almost two decades. Besides TeX/LaTeX he likes to work with Python, Linux, Rasberry/Arduino and his digital cameras.

More Posts - Website

Generating a normal distribution table with SciPy

Here’s a simple example how one can generate a normal distribution table with Python and scipy so that it can be imported into LaTeX.

Example-03.zip

# -*- coding: utf-8 -*-
"""
Created on Mon Mar 13 21:14:17 2017
@author: Uwe Ziegenhagen, ziegenhagen@gmail.com
 
Creates a CDF table for the standard normal distribution
 
use booktabs package in the preamble and put 
the generated numbers inside (use only one backslash!)
 
\\begin{tabular}{r|cccccccccc} \\toprule
<output here>
\\end{tabular}
"""
 
from scipy.stats import norm
 
print(norm.pdf(0))
print(norm.cdf(0),'\r\n')
 
horizontal = range(0,10,1)
vertikal = range(0,37)
 
header = ''
for i in horizontal:
    header = header + '& ' + str(i/100)
 
print(header, '\\\\ \\midrule')
 
for j in vertikal:  
    x = j/10
    print('\\\\', x)
    for i in horizontal:
        y = x + i/100
        print('& ', "{:10.4f}".format(norm.cdf(y),4))
 
 
print('\\\\ \\bottomrule \r\n')

Uwe

Uwe Ziegenhagen has been working with LaTeX for almost two decades. Besides TeX/LaTeX he likes to work with Python, Linux, Rasberry/Arduino and his digital cameras.

More Posts - Website

Computing Confidence Intervals with R

To highlight how confidence intervals cover in 1-\alpha cases the true parameter I hacked together a little R code based on code from http://paleocave.sciencesortof.com/2013/03/writing-a-for-loop-in-r/ and http://www.cyclismo.org/tutorial/R/confidence.html

  • I define a function which takes as parameter „runs“ the number of confidence intervals to compute
  • result is the vector of results, it will contain TRUE values for those intervals that cover the true parameter and FALSE for those which don’t
  • for each run I draw a sample of 10000 points from N(0,1)
  • I then calculate the mean and standard deviation of the sample
  • before calculating the limits of the confidence interval, „left“ and „right“
  • Since I know the true mu is 0 I can check if 0 falls into this interval, the result of this check is stored in the i-th column of the result vector
  • finally I calculate the summary of the result vector. On average 95% percent of all intervals will cover the true mu, 0.

If I find some more time I will add some functionality to run this code on multiple cores as well as a graphical visualisation of the intervals.

confInt <- function(runs){
	result<-NULL
	for (i in 1:runs) {
		data<-rnorm(10000)
		n<-length(data)
		a<-mean(data)
		s<-sd(data)
		error <- qnorm(0.975)*s/sqrt(n)
		left <- a-error
		right <- a+error
		result[i] = left<0 & 0<right
	}
	result
}
summary(confInt(100))

EDIT: Using some more ggplot2 code I have the graphical visualization ready:

confInt <- function(runs){
	x<-1:runs
	mu<-NULL
	sigma<-NULL
	result<-NULL
	vleft<-NULL
	vright<-NULL
 
	for (i in 1:runs) {
		data<-rnorm(1000)
		n<-length(data)
		a<-mean(data)
		mu[i]<-a
		s<-sd(data)
		sigma[i]<-s
		error <- qnorm(0.975)*s/sqrt(n)
		left <- a-error
		right <- a+error
 
		result[i] = left<0 & 0<right
		vleft[i] = left	
		vright[i] = right
}
	data.frame(x,mu,sigma,result,vleft,vright)
}
 
 
df<-confInt(100)
 
require(ggplot2)
 
myplot<-ggplot(df, aes(x = x, y = mu)) +
  geom_point(size = 2) +
  geom_errorbar(aes(ymax = vleft, ymin = vright,colour=result*3))
myplot + theme_bw() 
summary(df)

See http://stackoverflow.com/questions/30289894/plotting-confidence-intervals-from-a-dataframe/30290123#30290123 for an alternative solution.

Uwe

Uwe Ziegenhagen has been working with LaTeX for almost two decades. Besides TeX/LaTeX he likes to work with Python, Linux, Rasberry/Arduino and his digital cameras.

More Posts - Website

Neue Version der Herleitung der Linearen Regression verfügbar

Nach Hinweisen auf einen kleinen Fehler (danke Axel!) in meiner Herleitung der Gleichung für die lineare Regression habe eben eine neue Version hochgeladen: linreg.pdf.

Zusätzlich zur Fehlerbeseitigung habe ich mehr erläuternde Kommentare und Formeln hinzugefügt. Selbstredend wurde das Dokument in LaTeX gesetzt; es ist auch ein schönes Beispiel, was mit LaTeX & Co geht:

– die Grafik wurde in Metapost erstellt (heute würde ich sicher TikZ nehmen)
– alle Quelldateien sind im PDF-Container hinterlegt und verlinkt.

Wenn ich mal passenden R-Code finde (Hinweise werden gern entgegen genommen) werde ich noch mehr Beispiele für die Auswirkungen von Anstieg und Achsenabschnitt der Regressionsgeraden auf die Quadratsumme einfügen.

Uwe

Uwe Ziegenhagen has been working with LaTeX for almost two decades. Besides TeX/LaTeX he likes to work with Python, Linux, Rasberry/Arduino and his digital cameras.

More Posts - Website

T-SQL: Median und Ausreißer berechnen

Unter http://www.insidesql.org/blogs/frankkalis/2004/07/13/median-berechnen und http://quantmeditate.blogspot.com/2005/03/computing-interquartile-range.html habe ich Informationen zur Berechnung von Median und Quartilen in T-SQL gefunden.

DROP TABLE #data
CREATE TABLE #data(NUMBER FLOAT)
 
INSERT INTO #data VALUES(1.0);INSERT INTO #data VALUES(2.0);
INSERT INTO #data VALUES(3.0);INSERT INTO #data VALUES(4.0);
INSERT INTO #data VALUES(5.0);INSERT INTO #data VALUES(6.0);
INSERT INTO #data VALUES(7.0);INSERT INTO #data VALUES(8.0);
INSERT INTO #data VALUES(9.0);INSERT INTO #data VALUES(10.0);
INSERT INTO #data VALUES(11.0);INSERT INTO #data VALUES(12.0);
INSERT INTO #data VALUES(13.0);INSERT INTO #data VALUES(14.0);
INSERT INTO #data VALUES(15.0);INSERT INTO #data VALUES(16.0);
INSERT INTO #data VALUES(17.0);INSERT INTO #data VALUES(18.0);
INSERT INTO #data VALUES(19.0);INSERT INTO #data VALUES(20.0);
INSERT INTO #data VALUES(21.0);INSERT INTO #data VALUES(22.0);
INSERT INTO #data VALUES(23.0);INSERT INTO #data VALUES(24.0);
INSERT INTO #data VALUES(25.0);INSERT INTO #data VALUES(26.0);
INSERT INTO #data VALUES(27.0);INSERT INTO #data VALUES(28.0);
INSERT INTO #data VALUES(29.0);INSERT INTO #data VALUES(30.0);
INSERT INTO #data VALUES(31.0);INSERT INTO #data VALUES(32.0);
INSERT INTO #data VALUES(33.0);INSERT INTO #data VALUES(34.0);
INSERT INTO #data VALUES(35.0);INSERT INTO #data VALUES(36.0);
INSERT INTO #data VALUES(37.0);INSERT INTO #data VALUES(38.0);
INSERT INTO #data VALUES(39.0);INSERT INTO #data VALUES(40.0);
INSERT INTO #data VALUES(-100.0);INSERT INTO #data VALUES(100.0);
 
DECLARE @median FLOAT
DECLARE @perc25 FLOAT
DECLARE @perc75 FLOAT
DECLARE @iqr FLOAT
 
SET @median = (SELECT   MAX(NUMBER) AS Median FROM (SELECT TOP 50 PERCENT NUMBER FROM #data ORDER BY NUMBER) a)
SET @perc25 = (SELECT   MAX(NUMBER) AS Median FROM (SELECT TOP 25 PERCENT NUMBER FROM #data ORDER BY    NUMBER) a)
SET @perc75 = (SELECT   MAX(NUMBER) AS Median FROM (SELECT TOP 75 PERCENT NUMBER FROM #data ORDER BY    NUMBER) a)
 
print @median
print @perc25
print @perc75
 
SET @iqr = @perc75-@perc25
print 'IQR'
print @iqr
 
SELECT * FROM #data WHERE NUMBER <  (@median + 1.5* @iqr) AND 
NUMBER >  (@median - 1.5* @iqr)
 
print 'number of outliers'
SELECT COUNT(*) FROM #data WHERE NUMBER <  (@median - 1.5* @iqr)
OR NUMBER >  (@median + 1.5* @iqr)

Uwe

Uwe Ziegenhagen has been working with LaTeX for almost two decades. Besides TeX/LaTeX he likes to work with Python, Linux, Rasberry/Arduino and his digital cameras.

More Posts - Website

Open Data: LibraryHack 2011 in Australien

Im Mai 2011 findet der „LibraryHack 2011“ in Australien statt, bei dem die Teilnehmer aus öffentlichen Daten neue und nützliche Informationen generieren sollen. Mehr Informationen dazu unter der folgenden URL: http://data.gov.au/tag/libraryhack-2011/.

Ein interessanter Datensatz ist z.B. der zu den britischen Verurteilten, die im Zeitraum 1787-1867 nach Australien deportiert wurden: http://data.gov.au/dataset/british-convict-transportation-registers/.

Uwe

Uwe Ziegenhagen has been working with LaTeX for almost two decades. Besides TeX/LaTeX he likes to work with Python, Linux, Rasberry/Arduino and his digital cameras.

More Posts - Website

Open Data: Australische Verwaltungsdaten im Internet

Golem hatte vor ein paar Tagen eine Meldung, dass Australien Verwaltungsinformationen zur freien Verfügung herausgibt. Hier der Link zu Meldung: http://www.golem.de/1103/82037.html.

Uwe

Uwe Ziegenhagen has been working with LaTeX for almost two decades. Besides TeX/LaTeX he likes to work with Python, Linux, Rasberry/Arduino and his digital cameras.

More Posts - Website

Open Data: Freigabe bayerischer Luftbilder

Golem hat vor ein paar Tagen gemeldet, dass der Freistaat Bayern Luftbilder mit einer 2m-Auflösung freigibt. Hier der Link zum Artikel: (http://www.golem.de/1103/82039.html)

Uwe

Uwe Ziegenhagen has been working with LaTeX for almost two decades. Besides TeX/LaTeX he likes to work with Python, Linux, Rasberry/Arduino and his digital cameras.

More Posts - Website

Open Data Treffen in Köln

Am 08.01.2011 fand im CoWoCo eine Veranstaltung zum Thema „Open Data / Open Government“ statt. Aus einer der Sessions ging die Idee hervor, eine einführende Präsentation zu erstellen, was „Open Data“ ist und wofür es gut ist.

Die Präsentation lege ich unter einer eigenen Seite, http://uweziegenhagen.de/?page_id=1214 ab.

Uwe

Uwe Ziegenhagen has been working with LaTeX for almost two decades. Besides TeX/LaTeX he likes to work with Python, Linux, Rasberry/Arduino and his digital cameras.

More Posts - Website

Artikel zu R in der NY Times

Anfang 2009 war ein interessanter Artikel zu R in der NY Times: http://www.nytimes.com/2009/01/07/technology/business-computing/07program.html?_r=3.

Uwe

Uwe Ziegenhagen has been working with LaTeX for almost two decades. Besides TeX/LaTeX he likes to work with Python, Linux, Rasberry/Arduino and his digital cameras.

More Posts - Website