README.dev

This file is for those interested in developing DBD::Pg. It is hoped that it 
will be a good introduction as well as a continual reference. Suggestions 
are always welcome.

Note: most of this document assumes you are using a Unix-like system.

Sections:

* Overview
* File List
* Compiling
* Editing
* Heavy Testing
* Debugging
* Test Files
* Version Numbers
* New Files
* New Methods
* Making a New Release
* Tips and Tricks
* Resources
* Ongoing maintenance

==============
== Overview ==
==============

How It All Works

DBD::Pg is a combination of Perl, C, and XS, using files from the dbdpg project, 
the DBI module, and libpq - the C library interface to the PostgreSQL server. 
There is a sometimes complex interweaving of files needed for each method.

Running "perl Makefile.PL" uses the ExtUtils::MakeMaker module to create a 
true Makefile. Then the "make" command compiles everything, after creating 
the Pg.c file from Pg.xs and from the Perl.xsi file from DBI. The files Pg.pm and
blib/arch/auto/DBD/Pg/Pg.so form the core of the module once installed.
(The above is oversimplified).

The canonical git repo is at git://github.com/bucardo/dbdpg.git

===============
== File List ==
===============

Here is what each file in the distribution does:


* Text files:

Changes - lists changes made to each version. Please be consistent and use 
  spaces, not tabs, to indent. Try to list who found the bug, and who 
  fixed it (if not the same person). Put the CPAN or Github ticket number 
  in parenthesis, and put the person who made the actual changes in brackets. 
  When in doubt, copy the existing format. Note that t/00_release.t does 
  some simple format checking.
  This file contains a version number.

README.dev - you are reading it.

README - the main file that explains the module, where to get it, and guides 
  people in installing it. A large portion of it is simply a list of common 
  gotchas and guides for various platforms. This file has a version number 
  in it (or two, if this is a beta/release candidate)

README.win32 - the directions on how to install DBD::Pg on a Win32 box. Very likely 
  to be outdated.

README.testdatabase - created by the tests to cache connection information.
  Can be removed at will.

CONTRIBUTING.md - Quick introduction on how to contribute.

TODO - Rough list of possible items to fix or add.

SIGNATURE - Checksum verification via PGP, generated by Module::Signature.

LICENSES/gpl-2.0.txt - GPL license

LICENSES/artistic.txt - Artistic (Perl) license

testme.tmp.pl - Quick helper file for testing individual bugs

.dir-locals.el - Emacs helper file

* Build files:

Makefile.PL - The main file that starts everything off. Used by ExtUtils::MakeMaker 
  to create the "Makefile". This file contains a version number.

Makefile - Generated automatically by Makefile.PL. Not part of the distribution.

META.yaml - YAML description file. Updated by hand and contains a version number in three places.

META.json - JSON description file. Updated by hand and contains a version number in three places.

lib/Bundle/DBD/Pg.pm - Simple file used to enable perl -MCPAN -e 'install Bundle::DBD::Pg'
  Contains a version number.

MYMETA.yml, MYMETA.json - Files automatically created: can be ignored.


* Distribution files:

MANIFEST - lists which files should be included in the release tarball. 
  Used by the "make dist*" set of commands.

MANIFEST.SKIP - files that are known to be safe to exclude from the release 
  tarball. Used by the "make dist", "make distcheck" and "make skipcheck" commands.

win32.mak - a helper file for the win32 build.


* Program files:

dbdimp.c - The main C file, which does most of the heavy lifting for the DBD::Pg 
  module (the rest is done by Pg.pm). Almost all of the complexity and power of 
  the module is here.

dbdimp.h - Header file for dbdimp.c.

dbivport.h - DBI portability macros. This should be the latest version from the 
  DBI git repository.

Pg.pm - The main Perl file, which contains DBD::Pg packages and code for the 
  methods. Often times code here calls code from Pg.xs and dbdimp.c.
  This file contains a version number in four places (twice in the code, twice 
  in the POD). The main documentation for the module lives here, as POD information.

Pg.xs - The Perl "glue" file for DBD::Pg. This file basically tells Perl how 
  to handle various methods. It makes many calls to dbdimp.c

Pg.c - Not part of the distribution, but created from Pg.xs as part of the 
  build process. Never edit this directly.

Pg.h - Header file for Pg.xs (and thus Pg.c)

quote.c - Various methods to help quote and dequote variables. Some of this is 
  now done on the backend, but it is still needed to support older versions 
  of PostgreSQL.

quote.h - Header file for quote.c

types.c - Lists all known data types for PostgreSQL.
  Can be run as a perl script to check for new types; rewrites the following:
  types.h types.c Pg.xs Pg.pm t/01constants.t 99_pod.t

types.h - Header file for types.c


* Test files:

dbdpg_test_postgres_versions.pl - Run the test suite against multiple versions of
  Postgres, both for compilation and target. Run with "--setup head,9.4,12" etc.
  to create Postgres directories in ~/pg, then run with no arguments to test all
  combinations. Specify one or more compilation versions with "-c" and one or more
  target versions with "-r". For example, "-c 9.4,11 -r head"

t/dbdpg_test_setup.pl - Common connection, schema creation, and schema destruction subs.
  Goes through a lot of trouble to try and get a database to test with.

t/00_release.t - Quick check that all version numbers match, some other sanity checks.

t/00basic.t - Very basic test to see if DBI and DBD::Pg load properly. Requires Test::Warn 
  for the version warning test.

t/00_signature.t - Uses Module::Signature to verify SIGNATURE file. All tests are skipped 
  if the environment variable TEST_SIGNATURE is not set.

t/01connect.t - Basic connection tests, outputs pretty, detailed connection information.

t/01constants.t - Quick test of pg_types.

t/02attribs.t - Tests all attributes.

t/03dbmethod.t - Tests all database handle methods.

t/03smethod.t - Tests all statement handle methods.

t/04misc.t - Tests tracing, data_sources, $DBDPG_DEFAULT, and txn status changes.

t/06bytea.t - Tests bytea manipulation.

t/07copy.t - Tests COPY-related methods.

t/08async.t - Tests asynchronous methods.

t/09arrays.t - Tests array manipulation.

t/10_pg_error_field.t - Tests $dbh->pg_error_field function

t/12placeholders.t - Tests placeholders.

t/20savepoints.t - Test savepoints.

t/30unicode.t - Test Unicode. Or at least UTF8.

t/99cleanup.t - Removes anything we have created for the tests (e.g. tables)

t/99_lint.t - Various minor code cleanliness checks. Requires TEST_CRITIC to be set.

t/99_perlcritic.t - Uses Perl::Critic to check Pg.pm and all of the test files.
  Requires that TEST_CRITIC is set. It is recommended that you get all the 
  Perl::Critic policies via Bundle::Perl::Critic::IncludingOptionalDependencies.

.perlcriticrc - Used by the above: we assume tests are run from the main dir.

t/99_pod.t - Verifies the POD of Pg.pm. Requires Test::POD version 0.95, and 
  Test::Pod::Coverage 1.04.

t/99_yaml.t - Uses Test::YAML::Meta to verify the META.yaml file.

t/99_spellcheck.t - Checks the spelling of everything it can.

dbdpg_test_database - May be created by the test suite as a place to store 
a new database cluster.

* Helper files

The module App::Info is inside the t/lib directory (we put it there to prevent CPAN 
from indexing it). It is used by Makefile.PL to determine the version of PostgreSQL 
we are compiling against (by calling pg_config). It consists of:
t/lib/App/Info.pm
t/lib/App/Info/Handler.pm
t/lib/App/Info/Handler/Prompt.pm
t/lib/App/Info/Handler/Print.pm
t/lib/App/Info/RDBMS.pm
t/lib/App/Info/RDBMS/PostgreSQL.pm
t/lib/App/Info/Request.pm
t/lib/App/Info/Util.pm


===============
== Compiling ==
===============

Compiling is generally done with gcc. However, we also need to support a wide variety 
of compilers. Things which may only cause a minor warning when using gcc may stop 
other compilers cold. One way to catch this early is to add some warning flags to gcc. 
This can be done by extending the $comp_opts string inside of the Makefile.PL file. 
There are many warnings that can be enabled (see the man page for gcc for the list). 
Some of these warnings trigger for things outside of our control, such as the code 
for DBI or Perl itself. You can define the environment variable DBDPG_GCCDEBUG to turn 
many of these options on automatically.

Within each section, the order is the same as found in man gcc.

## These are warnings that should only generate errors that we can fix:
$comp_opts .= " -Wchar-subscripts -Wcomment";
$comp_opts .= " -Wformat=2"; ## does -Wformat,-Wformat-y2k,-Wformat-nonliteral,-Wformat-security
$comp_opts .= " -Wnonnull";
$comp_opts .= " -Wuninitialized -Winit-self"; ## latter requires the former
$comp_opts .= " -Wimplicit"; ## does -Wimplicit-int and -Wimplicit-function-declaration
$comp_opts .= " -Wmain -Wmissing-braces -Wparentheses -Wsequence-point -Wreturn-type -Wswitch -Wswitch-enum -Wtrigraphs";
$comp_opts .= " -Wunused"; ## contains -Wunused- function,label,parameter,variable,value
$comp_opts .= " -Wunknown-pragmas -Wstrict-aliasing";
$comp_opts .= " -Wall"; ## all of above, but we enumerate anyway
$comp_opts .= " -Wextra -Wendif-labels -Wpointer-arith";
$comp_opts .= " -Wbad-function-cast -Wcast-qual -Wcast-align -Wconversion -Wsign-compare -Waggregate-return";
$comp_opts .= " -Wmissing-prototypes -Wmissing-declarations -Wmissing-format-attribute -Wpacked -Winline -Winvalid-pch";
$comp_opts .= " -Wdisabled-optimization";
$comp_opts .= " -Wnested-externs";
$comp_opts .= " -Wstrict-prototypes"; ## Still hits a couple places in types.h
$comp_opts .= " -Wswitch-default";
$comp_opts .= " -Wsystem-headers";
$comp_opts .= " -Wmissing-noreturn";
$comp_opts .= " -Wfloat-equal"; ## Does not like SvTRUE() calls

## These options tend to produce lots of hits outside of our code, but may still be useful:
$comp_opts .= " -Wpadded"; ## Use when adding/changing our structs
$comp_opts .= " -pedantic"; ## Useful, but very verbose
$comp_opts .= " -Wundef"; ## Complains of XSubPPtmpAAAA in Pg.c being defined/undefined but then checked raw
$comp_opts .= " -Wshadow"; ## lots of bogus hits - not very useful Filter: grep warning wfile | grep -v "/usr"
$comp_opts .= " -Wwrite-strings";
$comp_opts .= " -Wredundant-decls"; ## Lots of warnings from Perl itself

## These options are probably not very useful:
$comp_opts .= " -Wtraditional"; ## Lots and lots of junk
$comp_opts .= " -Wold-style-definition"; ## We use lots of these
$comp_opts .= " -Wunreachable-code"; ## Lots due to our multi-version ifdefs

Please feel free to add to and clarify the above lists.


=============
== Editing ==
=============

All the perl files should have a cperl pragma at the top of the file, for easy use in emacs. 
Please use the same whitespace as surrounding code, and keep the indenting to the cperl standard.
Use the traditional C mode for *.c files. Pg.xs is a special case: if you know of a good 
mode for editing this file, please let us know and update this paragraph!

Please follow the other syntax standards in place as much as possible. A few guidelines 
for XS files can be found in the XS perldocs. When in doubt, go with the guidelines from 
Damian Conway's Perl Best Practices book.


===================
== Heavy Testing ==
===================

Testing should be done heavily and frequently, especially before a new release. 
The standard way to test is run "make test" which runs all the scripts in the 
"t" directory. If you find yourself making your own test, even if just for a 
minor or a temporary problem, please add it to the test suite. The more tests 
we have, the better.

Generally, we want to run 'make test' on as wide a variety of configurations as 
possible. If you have different platforms of course, you should test all of those. 
Beyond that, you may find it helpful to set up some aliases to allow quick switching 
of Postgres and DBI versions. You should generally test each major version of PostgreSQL 
that DBD::Pg currently supports. Keep in mind that there are two things to test for each 
version: the server that we are compiling against (e.g. which libraries we are linking to) 
and the version we are connecting to. You should test all variations.

One way is to keep multiple versions of PostgreSQL in standard directories, and use 
a standard port convention to keep things simple: the port is 5XXX where XXX is the 
version, so that PG 7.4.2 is listening on port 5742. Then set up two aliases for each 
version, like so:

alias dbd747='export DBI_DSN="dbi:Pg:dbname=greg;port=5747"'
alias dbd747m='export POSTGRES_LIB=/home/greg/pg747/lib POSTGRES_INCLUDE=/home/greg/pg747/include POSTGRES_DATA=/home/greg/pg747'

This allows for quick testing of each combination:

> dbd747m
> dbd747
> perl Makefile.PL
> make test (check output for any errors)
> dbd739
> make test
> dbd802
> make test
> dbd739m
> perl Makefile.PL
> make test
> dbd727
> make test
> dbd802
> make test

etc...

It's also a good idea to test the current HEAD version of Postgres in your tests: 
this can detect changes nice and early.

See the dbdpg_test_postgres_versions.pl file for one way to automate this.

In addition to different versions of Postgres, it's a good idea to test a few versions of 
DBI: this has caught problems in the past. You'll basically need to install the different 
versions of DBI into different directories, then adjust PERL5LIB with aliases:

alias dbi156='export PERL5LIB=/home/greg/perl/dbi156/lib/perl5/site_perl/5.10.0/i686-linux'
alias dbi157='export PERL5LIB=/home/greg/perl/dbi157/lib/perl5/site_perl/5.10.0/i686-linux'

Different encoding should also be tested: a good one for flushing out problems is BIG5, as 
it is not supported as a server encoding, only a client one. The simplest way to do this 
is to export the PGCLIENTENCODING variable to 'BIG5' before running the tests.

* Using splint

Another great program to use is splint, which is a "tool for statically checking C programs for 
security vulnerabilities and common programming mistakes." It can be found at

http://www.splint.org/

It is typically run against a single C file, in our case, dbdimp.c and the generated Pg.c file. 
This is a very finicky tool. There is a "splint" target in the Makefile. There are three 
challenges to using splint:

1) Getting it to work in the first place. As the Makefile.PL section says, you need at least 
version 3.1.2. You also need to include all the relevant files, which Makefile.PL should do 
for you. Note that 'make splint' expects the TMP environment variable to be set to a writeable 
directory.

2) Limiting the amount of results. splint is extremely verbose, so one must usually limit 
what sort of things are returned. Again, the Makefile.PL has a partial list.

3) Figuring out the real problems. Again, splint's verbosity takes some getting used to, as
does interpreting its output, and deciding what is really a problem and what is not.

* Using valgrind

We've not gotten valgrind to work against DBD::Pg, but would love to. Please email the list 
if you manage to do so!

* Using Devel::Cover

Another handy tool is the module Devel::Cover. While not totally useful as it only tests 
direct perl modules, it is good at giving Pg.pm the once-over. To use, install it, then run:

cover -delete
HARNESS_PERL_SWITCHES=-MDevel::Cover make test
cover -coverage statement,branch,condition,subroutine

then check out the coverage.html file inside the cover_db directory.

* Using Devel::DProf

This module is good for finding bottlenecks in the C portion of the code. Generally, you 
create a small test file that does heavy looping over some methods of interest, and 
then run it with:

perl -d:DProf testfile.pl

Then view the results with:

dprofpp

* Using Devel::NYTProf

Another nice Perl-level profiler. To use:

perl -d:NYTProf testfile.pl

Then run:

nytprofhtml

and check out the generated HTML files.


===============
== Debugging ==
===============

In addition to the Heavy Testing section, there are some simple aids to debugging.

* Testing file

It is helpful to have a standard file (e.g. ping.test.tmp) which contains some connection 
information and allows to easily stick in a piece of code for testing. It should run 
"make" to make sure everything is up to date. Here's the top of one such file:

#!perl -w

BEGIN {
	my $out = `make 2>&1`;
	if ($out =~ /^\w+\.[cx]s?:\d+:/ms or $out =~ /^Error/ms) {
		for (split /\n/ => $out) {
			print "MAKE ERROR: $_\n" if /^[\w\.]+:/;
		}
		exit;
	}

	use lib ".", "blib/lib", "blib/arch";
}
END {	print "End ping.test\n"; } BEGIN {	print "Begin ping.test\n"; }

use strict; use warnings; use Data::Dumper; $Data::Dumper::Deepcopy=1; use DBD::Pg; use DBI qw(:sql_types);
$|=1; select((select(STDERR),$|=1)[0]);
our ($dbh, $SQL, $sql, $sth, $count, $version, $info, $result, $id, $val);
my $trace = shift || 0;
my $dv = $DBI::VERSION; print "DBI version: $dv\n";
my $pv = $DBD::Pg::VERSION; print "DBD::Pg version: $pv\n";
my $DSN = $ENV{DBI_DSN};
$dbh = DBI->connect($DSN, $ENV{DBI_USER}, '', {AutoCommit=>0, RaiseError=>1, PrintError=>1});
my $VER = $dbh->{pg_server_version}; my $pgver = $dbh->{pg_lib_version};
print "Connected to $DSN\nServer version: $VER\nCompiled version: $pgver\n";
$dbh->trace($trace);

__END__

Once you have completed a test, just put it below the __END__ line in case you ever need to 
use it again someday. Note that the first argument to this script is the trace level. 
Bumping the trace level to 10 can be very helpful. If it is not helpful, consider adding 
some debugging statements to dbdimp.c to make it so!

* Coredumps

If you get a coredump, you can use the "gdb" utility to see what happened. Here's a 10-second 
tutorial. If "core" is the name of the core file, just use "gdb perl core", then issue a 
"bt" command at the gdb prompt. This will run a backtrace and give you an idea of what is 
causing the problem.

* For really low-level debugging from the Postgres side, you can use pg_server_trace() function.

* The perl debugger can also be helpful (perl -d ping.test.tmp).

* Don't forget about the PostgreSQL server logs either, when investigating matters.


================
== Test Files ==
================

The test files are an important part of the module. Much work has gone into making 
the tests as complete, thorough, and clean as possible. Please try to follow these 
guidelines when developing:

* Whenever you add a new feature, no matter how minor, add a test. Better yet, add 
many tests to make sure that it not only works correctly, but that it breaks when 
it is supposed to (e.g. when it is fed the wrong output). Try to conceive of every 
possible way your feature will be used and mis-used. Consider the effects of older 
versions of Perl, DBI, and/or Postgres.

* If someone files a bug report that is not revealed by a test, please add a new test 
for it, no matter how simple the fix maybe, or how stupid the bug is.

* Don't create a new test file unless necessary - use the existing ones whenever possible. 
Most things can fit in 03dbmethod.t (database handle methods) or 03smethod.t (statement 
handle methods). If all else fails, consider using the 04misc.t test. New files should 
generally be created for a bunch of related tests that do not easily fit into the current 
listings.

* If you do create a new test, keep the name short, start it with a number, and use an 
existing test as a template.

* Tests should be as "standalone" as possible. Most will call dbdpg_test_setup.pl to 
automatically setup the test table used. It's a good idea to delete any objects 
your test itself creates. Objects should be created as "temporary" whenever possible.
Things should be always have a name starting with "dbd_pg_test".

* Don't call DBI->connect inside of your tests, but use connect_database() from the 
dbdpg_test_setup.pl file instead. If you don't want it to blow away and recreate the 
current test table and other objects, use connect_database({nosetup => 1}).

* Use the standard format for tests, and always provide an appropriate output text. 
Abbreviations are encouraged, but be consistent throughout the file.

* Make sure to test on different versions of PostgreSQL, DBI, and Perl. Use the SKIP 
tag with an appropriate message if a test does not work on a particular version of 
something (see 20savepoints.t for an example).

* To run a single test, use: prove --blib . -v t/testname.t


=====================
== Version Numbers ==
=====================

Version numbers follow the older Postgres convention: major, minor, and revision. (Note: older versions 
of DBD::Pg used a two-number system up until version 1.49, after which it switched to 2.0.0).
The major number should very, very rarely change, and is saved for the truly major changes 
(e.g. those that may cause backwards compatibility problems). The minor revision is used to 
indicate a change in functionality, new features, etc. The revision number is used for small 
tweaks and bug fixes, and must be completely compatible with the version before it.

Beta versions (aka release candidates) are the version with an underscore at the end of it. The 
tells CPAN not to consider this a "real" release. For example, if the upcoming release is 2.2.4, 
the first release candidate would be 2.2.3_1. A second would be 2.2.3_2 etc.

Version numbers are currently set in seven files:

README (one place, two if a beta version)
Pg.pm (three places)
Changes
Makefile.PL
META.yaml (three places)
META.json (three places)
lib/Bundle/DBD/Pg.pm


===============
== New Files ==
===============

If you are adding a new file to the distribution (and this should be a rare event), 
please check that you have done the following items:

* Added it to git via 'git add filename' and 'git commit filename'

* Added it to the MANIFEST file

* Added it to Makefile.PL if needed, to make sure all build dependencies are met

* Updated/created necessary tests for it

* Added it to the "File List" section above.


=================
== New Methods ==
=================


New methods and attribute names should be short and descriptive. If they are "visible", 
please make sure they begin with a "pg_" prefix. If you feel they should not have this 
prefix, make your case on the dbi-dev list.


==========================
== Making a New Release ==
==========================

This is a comprehensive checklist of all the steps required to release a 
new version, whether beta or regular. It is assumed you are very familiar with the 
other sections referenced herein (indicated with **)

* Test on variety of versions (see ** Heavy Testing), including the optional tests.

* Test modules that depend on DBD::Pg, in particular DBIx::Class and DBIx::Class::Schema::Loader
  Do not forget to set DBICTEST_PG_DSN, DBICTEST_PG_USER, and DBICTEST_PG_PASS

* Consider a pre-release announcement to dbix-class-devel@lists.scsys.co.uk

* Make sure everything is up to date in git (git status)

* Update the versions (see ** Version Numbers) in README, Pg.pm (3 places!), Makefile.PL, 
  lib/Bundle/DBD/Pg.pm, META.yaml and META.json (3 places each), and Changes. Run the t/00_release.t file 
  to double check you got everything.

* If a final version, put the release date into the Changes file.

* If a beta version, please put a large warning at the top of the README file. Here is a sample:

===================================================
WARNING!!

THIS IS A TEST VERSION (2.4.1_2) AND SHOULD BE USED 
FOR TESTING PURPOSES ONLY. PLEASE USE A STABLE 
VERSION (no underscore) BY VISITING:

http://search.cpan.org/dist/DBD-Pg/
===================================================

* If not a beta version, remove the above warning from the README if it exists.

* Completely update the Changes file

The best way to do this (other than doing it as you go along) is to check the git logs, 
by running a diff against the last-released version.

* Update the documentation

Make sure that anything new has been documented properly, usually as POD inside of Pg.pm.
A good way to do this is to use the tests in 99_pod.t - they will run automatically as 
part of the test suite if the right modules are installed.

* Run "perl Makefile.PL"

* Run "make dist". Double check that the tarball created has the correct version name.

* Run "make distcheck". This will show you a list of all files that are in the current directory 
but not inside the MANIFEST file (or the MANIFEST.SKIP file). If there are any new files here 
that should be part of the distribution, add them to the MANIFEST file, commit your changes, 
and then re-run. Note that files ending in ".tmp" are currently skipped, so this is a good 
extension for any miscellaneous files you have that use often (e.g. libpq-fe.h.tmp)

* Run "make skipcheck". This will show you a list of files that will NOT be packed into the 
release tarball. Make sure there is nothing important here.

* Update the SIGNATURE file with Module::Signature (e.g. make signature)
You may need to add this your login script: export GPG_TTY=$(tty)

* Run "make disttest". This unpacks the tarball, then runs "make" and "make test" on it.
You may also want to remove the directory it creates later by using "make realclean"

* Make a new git tag: git tag -u 01234abc 1.2.3 -m "Version 1.2.3, released April 1, 2015"
In the example above, 01234abc is your pgp shortid and 1.2.3 is the new version number.
You might need: export GPG_TTY=$(tty)

* Make checksums

Generate md5 and sha1 checksums of the tarball. Include this in your emails.

* Test it out

Download the tarball to a completely different system, unpack and test it.

* Announce to the "internal" lists

dbd-pg@perl.org
pgsql-interfaces@postgresql.org

* Upload to CPAN and test.

You'll need the pause account password. The interface is fairly straightforward. Once it 
is loaded, wait for it to appear on the main DBD::Pg page and then test that the file 
has the same checksums.

* Commit the SIGNATURE file. Remember the git commit hash given, and add that 
to the Changes files. Then commit the Changes file.

* Announce to the "public" lists

dbi-users@perl.org, dbi-dev@perl.org, dbi-announce@perl.org

The format for DBI announcements:
To: dbi-announce@perl.org
Cc: dbi-users@perl.org
Reply-to: dbi-users@perl.org

Subject line: Name of module, version

Short note of changes, link to CPAN directory.

Checksums for the file.

See past announcements in the z_announcements directory.

* Post to pgsql-announce@postgresql.org if this is a major or important version.

* Post to the "PostgreSQL news"

On the main page, there is a link named "Submit News" which points to:

http://www.postgresql.org/about/submitnews

The content should be roughly the same as the announcement.

* PostgreSQL weekly news summary

The maintainers of the weekly news are usually pretty good about catching the update 
and adding it in. If not, bug them.

http://www.postgresql.org/community/weeklynews/

* Tell Greg to post on planet.postgresql.org

* If a non-beta, clean out any CPAN and github bugs, including going back and marking resolved 
bugs with this new version, once it appears in the choices (takes a day or two for 
the version to appear as a choice in the pulldown on CPAN).

* Check the CPAN testers report a few days after the PAUSE upload:
http://matrix.cpantesters.org/?dist=DBD-Pg

* Update this file based on your experiences!!


=====================
== Tips and Tricks ==
=====================

Also known as, the section to put things that don't fit anywhere else. Anything 
that may make life easier for other developers can go here.

* Temporary tables

We do not use temporary tables in most of our tests because they are 
not persistent across tests, they mess up the schema testing, and they 
are not compatible with the foreign key testing. But do try and use them 
whenever possible.

* "turnstep" in the cvs/svn/git logs is Greg Sabino Mullane, greg@turnstep.com.

* Use a "tmp" extension for files you keep around in the dbdpg directory, but 
don't want to show up when you do a "git status". They are also ignored by make dist.

* Commit each file individually, unless the log message is *really* identical 
across all committed files (which is rare). Always give a good description 
of the exact changes made : assume that the log will be read independently 
of a diff.

* Don't forget to test for memory leaks, particularly if you are working with 
the more complicated sections of dbdimp.c. For a quick check, enter a loop, 
then watch the memory size using the top tool. Here's a quick checker:

$dbh->{pg_server_prepare} = 1;
$dbh->{pg_direct} = 1;

$dbh->do("CREATE TEMP TABLE leaktester(a int, b numeric(10,2), c text)");
$sth{'plain'} = $dbh->prepare("SELECT * from leaktester");
$sth{'place'} = $dbh->prepare("INSERT INTO leaktester(a,b,c) VALUES (?,?,?)");
my $loop = 1;
while (1)
{

	$sth{plain}->execute;
	$dbh->do("SELECT 123");
	$dbh->quote(qq{Pi''zza!!"abc});
	$sth->{pg_server_prepare}=1;
	$sth{place}->execute(1,2,"abc");
	$sth->{pg_server_prepare}=0;
	$sth{place}->execute(1,2,"abc");
	$sth->{pg_server_prepare}=1;

	$sth = $dbh->prepare("SELECT 123, ?");
	$sth->bind_param(1,1,SQL_INTEGER);
	$sth->execute(1);
	$sth->finish();

	$info = $dbh->selectall_arrayref("SELECT 123,456");
	select(undef,undef,undef,0.1);
	exit if $loop++ > 10000;
}


===============
== Resources ==
===============

The primary resource is the mailing list, where the developers live. Other 
resources depend on the subject:

* DBD::Pg

The canonical URL:
http://search.cpan.org/dist/DBD-Pg/

* CPAN::Reporter test results:
http://matrix.cpantesters.org/?dist=DBD-Pg


* DBI

The DBI developers list:
http://lists.perl.org/showlist.cgi?name=dbi-dev

The DBI users list:
http://lists.perl.org/showlist.cgi?name=dbi-users

The DBI announcement list:
http://lists.perl.org/showlist.cgi?name=dbi-announce

The latest DBI:
http://search.cpan.org/dist/DBI/

The source code of other DBDs can be a useful tool as well.

* Postgres

A good source for general questions on libpq and similar things is the pgsql-hackers list. Having a 
copy of the Postgres source code is invaluable as well. Using a tool like glimpse or ctags is handy 
to find those obscure libpq functions quickly. You also may want to keep the libpq documentation handy.

All of the Postgres mailing lists:
http://www.postgresql.org/community/lists/

A great source for searching the pg documentation and mailing lists is:

http://www.pgsql.ru/db/pgsearch/

which allows you to limit the search by version: very helpful as we support multiple versions of PostgreSQL.

There are many ways to search the Postgres mailing lists:

http://postgresql.markmail.org/

http://archives.postgresql.org/

http://groups.google.com/ (add group:pgsql.*)


* Perl

Besides a good general understanding of Perl, it helps to learn a little bit about XS:

perldoc perlapi
perldoc perlclib
perldoc perlguts
perldoc perlxstut
perldoc perlxs

This is the module that does all the introductory magic:

perldoc ExtUtils::MakeMaker


The all important testing suite:

perldoc Test
perldoc Test::Harness
perldoc Test::Simple
perldoc Test::More
perldoc Test::Pod
perldoc Test::Pod::Coverage
perldoc Test::YAML::Meta


Other important modules:

perldoc Devel::Cover
perldoc Module::Signature
perldoc Perl::Critic
perldoc DBI::Profile


Also see perldoc DBI::DBD. It's fairly old and incomplete, but still useful.


=======================
= Ongoing maintenance =
=======================

Keeping a project healthy requires regular maintenance. Here are some things 
that need to be done regularly:

* Update the copyright years (yearly in January)

* Copy over dbivport.h from DBI and recompile (check monthly, or when a change is noticed)

* Run `perl -x types.c` against Postgres HEAD (often as possible, but once a month is fine)

* Check the bug reports from github, mailing lists, etc. (monthly)

* Test against latest version of DBI (upon new release, but also as often as possible)