* chapter_analysis.do
* ---------------------------------------------------------------------------
* Supplementary archive for "Entrepreneurship and Behavioral Characteristics"
* Dubard Barbosa, Cooper, Saral and Villeval
* Experiments in Entrepreneurship, Volume 1 (Edward Elgar, 2027)
*
* THIS FILE PRODUCES EVERY NUMBER IN THE CHAPTER.  Each block is labelled with
* the table or the passage it feeds.
*
* ---------------------------------------------------------------------------
* TO RUN IT
* ---------------------------------------------------------------------------
* Unzip the archive, open Stata in the archive folder, and run this file.  The
* DA global below already points at the data/ subfolder, so nothing needs
* changing.  Stata 18 SE was used.  Nothing is written except the log; the only
* "save" in the file is to a tempfile.
*
* NOTE ON THE COMMENTS BELOW.  The file keeps its original development history,
* including the authors' own change logs.  Those comments name files and folders
* in the project's working tree -- cleaningI.do, "source materials/...",
* "djc take 4.do" and so on.  Those are provenance citations recording where
* something came from.  THEY ARE NOT PART OF THIS DEPOSIT and no reader needs
* them; everything required to reproduce the chapter is in this archive.
*
* ---------------------------------------------------------------------------
* WHICH DATA PRODUCES THE PUBLISHED NUMBERS -- READ THIS
* ---------------------------------------------------------------------------
* No single dataset reproduces the chapter.  Three are used:
*
*   ent_djc_3.dta   the cross section
*   ent_djc_4.dta   the 2018 follow-up
*   ent_final2.dta  the Phase 3 (online experiment) measures only
*
* ent_djc_3.dta is dated 26 December 2017 and so PREDATES the second wave's
* online experiment, which ran in early 2018.  It carries 76 observations on
* the five online measures where 95 exist.  The program p3_fill below merges
* those ten variables in from ent_final2.dta (global P3FINAL = 1).  Expected
* result: 95 observations, 66 / 13 / 16 by entv2_c.
*
* The eyes test needs more than a merge.  numcorrect_eyes DIFFERS between
* ent_djc_3.dta and ent_final2.dta for 59 of the 76 subjects present in both,
* while every other online variable agrees exactly.  ent_final2.dta takes its
* scores from the raw oTree export; for the 47 French-language subjects these
* were re-scored item by item against the answer key in the oTree source
* (ENT5_fr/eyes_mind/models.py) and match exactly.  The origin of the
* ent_djc_3.dta values is not known.  numcorrect_eyes is therefore taken from
* ent_final2.dta for everyone (global P3REPLACE).
*
* => Running this file against ent_djc_3.dta ALONE will not reproduce the
*    chapter.  You will get 76 observations on the Phase 3 rows of Table 5 and
*    different eyes-test numbers.
*
* Analysis sample is 122.  Subjects 206 (2016) and 709 (2017) are the same
* person; the 2017 record is dropped.
*
* ---------------------------------------------------------------------------
* PROVENANCE
* ---------------------------------------------------------------------------
* This is David Cooper's "chapter_analysis (djc 2026) v5.do" of 4 October 2026,
* which is the version that generated the published tables, with two changes
* and no others: the data path is a single global below, and the log is renamed.
* His own header, recording every change he made from the 30 September version,
* follows immediately.
* ---------------------------------------------------------------------------

* chapter_analysis (djc 2026) v5.do
* Same as chapter_analysis (djc 2026).do plus the rebuilt Table 5, a
* second version of the lasso, the online-experiment (Phase 3) measures
* taken from ent_final2.dta, and the eyes-test scores replaced by the
* ent_final2.dta values (4 Oct 2026).
* David's working version of Krista's chapter_analysis.do (30 Sept 2026).
* Changes from Krista's file are listed here as they are made:
*   - log file renamed so it does not overwrite Krista's chapter_analysis.log
*   - request2 labelled, with a comment explaining how it is constructed
*   - 11-20 game: ordered probit now uses request (untruncated) rather than
*     request2, in two specifications, without and with skilltotalcorrect
*     (Raven's), as in djc take 4.do.  request2 versions kept, commented out.
*   - income control is i.broad_income_2 in every cross-section
*     specification.  Krista's file used i.broad_income for Table 3, market
*     entry and the Table 5 OLS screen; those now use i.broad_income_2.
*   - Table 4 and its Section 3.3 robustness loop: income control changed
*     from broad_income_2 entered linearly to i.broad_income_2.
*   - global DA points to David's Dropbox; Krista's path kept, commented out.
*   - follow-up response probit (Section 3.3): now uses the Table 3 controls,
*     i.entv2_c age female i.broad_income_2 i.broad_education.  Krista's
*     age-and-gender version kept, commented out.
*   - NEW TABLE block added after market entry: Table 3 column (1)
*     specification for all seven Table 2 outcomes, marginal effects at
*     means, collected in matrix NT.
*   - NEW TABLE: Wald test that both classification coefficients are zero
*     (chi2 with 2 df and p-value) added as rows 14-15 of matrix NT.
*   - NEW TABLE: difference between the entrepreneur and future-entrepreneur
*     marginal effects (entrepreneur - future entrepreneur), with
*     delta-method SE and p-value, added as rows 16-18 of matrix NT.
*   - Table 4 rebuilt: the 2018 risk and patience measures are dropped, and
*     all seven laboratory measures from Tables 2 and 3 are entered.  The
*     two trust measures are demeaned with missing (session 1) set to zero.
*     Coefficients and robust SEs collected in matrix T4, joint model and
*     one-measure-at-a-time models side by side.  Previous Table 4
*     specification kept, commented out.
*   - Table 4 column 3: joint model plus the 2016 survey risk measure
*     (fast_risk, NOT fast_risk18), demeaned with missing set to zero.
*     Diagnostics added: coverage of fast_risk among follow-up respondents
*     and the correlation of fast_risk with fast_risk18.
*   - Table 4 columns 4 and 5: average marginal effects from the column 1
*     joint model.  Column 4: effect on Pr(entrepreneur in 2018).  Column 5:
*     effect on Pr(future entrepreneur or entrepreneur in 2018).  Market
*     entry and tournament entry entered as factor variables so their
*     effects are discrete 0->1 changes; their coefficients are unchanged.
*     tab entv2_18 added to confirm the coding of 2018 status.
*   - Table 4 columns 6 and 7: columns 4 and 5 rescaled to the effect of a
*     one-standard-deviation change in each continuous measure (SD in the
*     estimation sample; for trust, among respondents with valid trust
*     data).  Market and tournament entry left as 0->1 changes.  SDs used
*     listed in matrix T4SD.
*   - Table 4 diagnostic: linear lasso on 2018 status with lambda chosen by
*     BIC, (a) controls always included and the seven measures penalized,
*     (b) everything penalized.  Not a table in the chapter.
*   - The seven-column Table 4 (matrix T4) is now a DIAGNOSTIC, not in the
*     paper.  The paper's Table 4 is matrix T4P, four columns: T4 columns
*     1, 6 and 2, plus a new ordered probit with all seven measures and
*     only baseline status and education as controls.
*   - TABLE 5 rebuilt (4 Oct 2026): every measure in the Table 5 OLS screen
*     (32 measures) re-estimated with the Table 3 specification -- same
*     controls, robust SEs, marginal effects at means, Wald test, and
*     entrepreneur - future entrepreneur difference.  Estimator follows the
*     outcome as in Table 3: ordered probit for single-item integer scales
*     and small counts, Tobit at the scale limits for averaged scales and
*     scores, probit for binary items.  All 32 in matrix T5ALL; the paper's
*     Table 5 is matrix T5, which keeps only measures with p < 0.10 on the
*     future-entrepreneur coefficient, the entrepreneur coefficient, or the
*     Wald test.  The OLS screen is kept, unchanged, for comparison.
*     WHERE TO FIND IT: the block headed "TABLE 5 -- PAPER VERSION", in the
*     CROSS SECTION part, immediately after the OLS screen and before the
*     survey-risk coverage tab.  It reuses $NTX and $NTME from the NEW
*     TABLE (Table 3) block, so run it after that block.  In the log,
*     search for "TABLE 5 (PAPER)" for the kept rows and "TABLE 5, ALL 32
*     MEASURES" for the full set.
*   - log file renamed to "chapter_analysis (djc 2026) v5.log" so this
*     version does not overwrite the previous versions' logs.
*   - PHASE 3 FIX (v4, 4 Oct 2026).  ent_djc_3.dta (Dec 2017) predates the
*     wave-2 online experiment (early 2018), so the five online measures
*     had 76 observations where 95 exist.  With global P3FINAL = 1 (set
*     just below global DA) the online variables -- risk1-risk4,
*     skewchoice1-skewchoice3, patience, playertails, numcorrect_eyes --
*     are filled in from ent_final2.dta (May 2018), merged on subject_id.
*     Everything else still comes from ent_djc_3.dta / ent_djc_4.dta.
*     Program p3_fill does the merge.  It first checks that the two files
*     agree for every subject who has the measure in both, and STOPS with
*     an error if any value differs.  Applied in the cross section (right
*     after the sample is set) and in the follow-up section (before lasso
*     version 2 builds its variables).  Expected result: 95 observations
*     (66/13/16) on each online measure in the cross section.  Affects
*     the online rows of Table 5, the fast_risk-eg_risk correlation and
*     the online-attrition tab in Section 3.4, and lasso version 2.  Set
*     P3FINAL to 0 to reproduce v3 (76 observations).
*   - EYES-TEST FIX (v5, 4 Oct 2026).  Running v4 stopped because
*     numcorrect_eyes differs between ent_djc_3.dta and ent_final2.dta for
*     59 of the 76 subjects in both.  Every other online variable agrees
*     exactly, so the online records are matched to the same people in
*     both files; only the eyes score differs.  ent_final2.dta takes its
*     scores from the raw oTree export (online.txt, imported by
*     cleaningI.do; source files in source materials/entrepreneurs/data/
*     Online Experiment/).  For the 47 French-language subjects in the
*     first online run, the exported scores were re-scored item by item
*     against the answer key in the oTree source
*     (ENT5_fr/eyes_mind/models.py) and match exactly.  The origin of the
*     ent_djc_3.dta values is not known.  Variables listed in global
*     P3REPLACE (numcorrect_eyes) are therefore taken from ent_final2.dta
*     for everyone, not just filled in; p3_fill reports how many values
*     change, their correlation with the old values, and lists each
*     changed subject before replacing.  Set P3REPLACE to "" to restore the
*     v4 behaviour (stop on any disagreement).
*   - LASSO VERSION 2 added (4 Oct 2026), directly after the original lasso
*     block in the FOLLOW UP part.  Same lasso (linear, BIC, t4s sample,
*     versions (a) and (b)) with the 32 Table 5 measures added to the
*     candidate set.  Those measures are demeaned with missing set to zero
*     (prefix l5_), no missing-data dummies.  Results stored as las_a2 and
*     las_b2; the final lassocoef compares all four lasso fits.  In the
*     log, search for "LASSO VERSIONS 1 AND 2".  Diagnostic, not in the
*     paper.
*
* ---- Krista's original header follows ----
*
* chapter_analysis.do
* Everything reported in "Entrepreneurship and Behavioral Characteristics",
* the Shane volume chapter.  Each block is labelled with the table or the
* passage in the chapter that it produces.
*
* Input is ent_djc_3.dta for the cross section and ent_djc_4.dta for the
* follow-up.  These are David's analysis files; the cleaning chain is not
* re-run.  Specifications follow djc take 3.do and djc take 4.do.
*
* Sample.  Subjects 206 (session 2, 2016) and 709 (session 7, 2017) are the
* same person.  We keep the first participation and drop 709, so the chapter
* uses 122 subjects and 66 follow-up responses where the raw data hold 123
* and 67.  David's saved tables use all 123.
*
* Krista Saral, 30 Sept 2026.  Stata SE 18.

clear all
set more off
set linesize 200
capture log close
log using "chapter_analysis.log", replace text

* >>> The folder holding ent_djc_3.dta, ent_djc_4.dta and ent_final2.dta.
* >>> In this archive they are in the "data" subfolder, so nothing needs
* >>> changing: unzip, open Stata in the archive folder, and run this file.
* >>> If you move the datasets elsewhere, point this at them instead.
* >>> A trailing slash is not needed.
global DA "data"

***************************************************************************
* PHASE 3 FIX (djc 2026, v4).  Online-experiment measures from ent_final2.dta
***************************************************************************
* 1 = fill the online measures in from ent_final2.dta (95 observations);
* 0 = use ent_djc_3.dta / ent_djc_4.dta only, as in v3 (76 observations).
global P3FINAL 1
global P3VARS "risk1 risk2 risk3 risk4 skewchoice1 skewchoice2 skewchoice3 patience playertails numcorrect_eyes"
* v5: online variables to take from ent_final2.dta for EVERYONE, replacing
* the values in memory.  The ent_djc_3.dta eyes scores disagree with the
* raw oTree data for 59 of 76 subjects (see the header).  "" = v4 behaviour.
global P3REPLACE "numcorrect_eyes"

* p3_fill: merge the online variables from ent_final2.dta on subject_id
* and fill in values missing from the data in memory.  For subjects who
* have a value in both files the values must agree; if any differ the
* program lists them and stops (error 459), so nothing is silently
* overwritten.  Exception (v5): variables in $P3REPLACE are replaced by
* the ent_final2.dta value for everyone who has one, after the
* differences are reported and listed.  ent_final2.dta is only read,
* never saved.
capture program drop p3_fill
program define p3_fill
    preserve
    use subject_id $P3VARS using "$DA/ent_final2.dta", clear
    foreach v of global P3VARS {
        rename `v' f2_`v'
    }
    tempfile p3
    save `p3'
    restore
    merge 1:1 subject_id using `p3', keep(master match) nogenerate
    foreach v of global P3VARS {
        capture confirm variable `v'
        if _rc {
            gen double `v' = .
        }
        local rep "$P3REPLACE"
        local isrep : list posof "`v'" in rep
        if `isrep' {
            display _newline as text "p3_fill: `v' is in P3REPLACE -- comparing, then taking ent_final2.dta values"
            quietly count if !missing(`v') & !missing(f2_`v')
            display as text "   subjects with `v' in both files: " r(N)
            quietly count if !missing(`v') & !missing(f2_`v') & abs(`v' - f2_`v') > 1e-6
            display as text "   of which the values differ: " r(N)
            if r(N) > 0 {
                quietly correlate `v' f2_`v' if !missing(`v') & !missing(f2_`v')
                display as text "   correlation, value in memory vs ent_final2.dta: " %6.3f r(rho)
                tempvar d
                quietly gen double `d' = `v' - f2_`v'
                summarize `d' if !missing(`d')
                list subject_id `v' f2_`v' if !missing(`v') & !missing(f2_`v') & abs(`v' - f2_`v') > 1e-6, noobs sep(0)
            }
            quietly count if missing(`v') & !missing(f2_`v')
            display as text "   added from ent_final2.dta (missing in memory): " r(N)
            quietly count if !missing(`v') & missing(f2_`v')
            if r(N) > 0 {
                display as text "   note: " r(N) " subjects have `v' in memory but not in ent_final2.dta; value in memory kept"
            }
            quietly replace `v' = f2_`v' if !missing(f2_`v')
            drop f2_`v'
            continue
        }
        quietly count if !missing(`v') & !missing(f2_`v') & abs(`v' - f2_`v') > 1e-6
        if r(N) > 0 {
            display as error "p3_fill: `v' differs between the data in memory and ent_final2.dta for " r(N) " subjects -- stopping"
            list subject_id `v' f2_`v' if !missing(`v') & !missing(f2_`v') & abs(`v' - f2_`v') > 1e-6
            exit 459
        }
        quietly count if !missing(`v') & missing(f2_`v')
        if r(N) > 0 {
            display as text "p3_fill: note, `v' present in memory but missing in ent_final2.dta for " r(N) " subjects; value in memory kept"
        }
        quietly count if missing(`v') & !missing(f2_`v')
        display as text "p3_fill: `v' filled in from ent_final2.dta for " r(N) " subjects"
        quietly replace `v' = f2_`v' if missing(`v')
        drop f2_`v'
    }
end

***************************************************************************
* CROSS SECTION
***************************************************************************

use "$DA/ent_djc_3.dta", clear
keep if !missing(entv2_c)
drop if repeat_2017   // the 2017 record of the repeat participant

* Phase 3 fix (v4): online measures from ent_final2.dta.  Check the
* counts: with P3FINAL = 1, expect 95 (66/13/16); with 0, 76 (48/12/16).
* v5: numcorrect_eyes is replaced by the ent_final2.dta values (P3REPLACE).
if $P3FINAL {
    display _newline "==== PHASE 3 FIX: filling online measures from ent_final2.dta ===="
    p3_fill
}
display _newline "==== online measures: observations by group (numcorrect_eyes) ===="
tab entv2_c if !missing(numcorrect_eyes)

egen eg_risk = rowmean(risk1 risk2 risk3 risk4)
egen eg_skew = rowmean(skewchoice1 skewchoice2 skewchoice3)
egen SE_all  = rowmean(search_SE plan_SE marshalling_SE implement_SE)

* request2 is a truncated version of request, the amount requested in the
* 11-20 game.  It is already in ent_djc_3.dta (created in the earlier
* cleaning code, not here).  Requests of 11 through 16 are pooled into a
* single category, coded 0; requests of 17 through 20 are coded as
* request - 16, i.e. 17 = 1, 18 = 2, 19 = 3, 20 = 4.  The pooling avoids
* sparsely populated low categories in the ordered probit.
* The chapter now uses the untruncated request throughout: Table 2 reports
* its mean, the ordered probits in Section 3.2 use it, and so do Table 4
* and the follow-up.  The request2 ordered probits are kept, commented out,
* in Section 3.2 for comparison.
label variable request2 "truncated version of request"

* classification of record.  79/16/28 on all 123, 78/16/28 here
tab entv2_c

* ---- Table 1.  subject characteristics ----
tabstat age female high_education high_income, by(entv2_c) stat(n mean) nototal

* ---- Table 2.  raw means, laboratory games ----
* trust excludes session 1, which ran a different version of the game
table entv2_c if session != 1, statistic(count give) statistic(mean give) statistic(mean ret_ave)
table entv2_c, statistic(mean contribution) statistic(mean cont_rate) statistic(mean request) ///
               statistic(mean enter) statistic(mean tournamentchoice)

* ---- Table 3.  tournament entry.  djc take 4.do ll. 155, 157 ----
* income control changed from i.broad_income to i.broad_income_2 (djc 2026)
xi: dprobit tournamentchoice i.entv2_c age female i.broad_income_2 i.broad_education, robust
xi: dprobit tournamentchoice i.entv2_c age female i.broad_income_2 i.broad_education correct1 correct2 dm_fast_risk, robust

* ---- Section 3.2.  trust, public goods, 11-20.  djc take 4.do ll. 125-140 ----
tobit give i.entv2_c age female i.broad_income_2 i.broad_education if session != 1, ll(0) ul(5) vce(robust)
tobit ret_ave i.entv2_c age female i.broad_income_2 i.broad_education if session != 1, ll(0) ul(1) vce(robust)
tobit contribution i.entv2_c age female i.broad_income_2 i.broad_education, ll(0) ul(20) vce(robust)
tobit cont_rate i.entv2_c age female i.broad_income_2 i.broad_education, ll(0) vce(robust)

* 11-20 game, ordered probit on the untruncated request (11-20).
* (1) baseline controls; (2) adds skilltotalcorrect, the number of Raven's
* matrices solved, for the "with or without controlling for performance on
* Raven's matrices" statement in Section 3.2.
oprobit request i.entv2_c age female i.broad_income_2 i.broad_education, robust
oprobit request i.entv2_c age female i.broad_income_2 i.broad_education skilltotalcorrect, robust

* Same two specifications on request2 (11-16 pooled as 0, 17-20 -> 1-4),
* as in djc take 4.do.  Not used in the chapter; kept for comparison.
* oprobit request2 i.entv2_c age female i.broad_income_2 i.broad_education, robust
* oprobit request2 i.entv2_c age female i.broad_income_2 i.broad_education skilltotalcorrect, robust

* ---- Section 3.2.  market entry.  djc take 4.do l. 147 ----
* income control changed from i.broad_income to i.broad_income_2 (djc 2026)
xi: dprobit enter i.entv2_c age female i.broad_income_2 i.broad_education, robust

* ---- NEW TABLE.  Model (1) for every Table 2 outcome, marginal effects (djc 2026) ----
* One regression per Table 2 measure, each with the Table 3 column (1)
* controls: classification, age, female, income (i.broad_income_2) and
* education, with robust standard errors.  The estimator follows the outcome:
*   (1)-(4) Tobit, censored at the game's limits  -> effect on E[y]
*   (5)     ordered probit on request (11-20)      -> effect on E[request]
*   (6)-(7) probit                                 -> effect on Pr(y = 1)
* All effects are evaluated at the covariate means (atmeans), as dprobit
* does for Table 3: a discrete 0->1 change for the two entrepreneur
* categories and for female, the derivative for age.  age and female are
* entered as c.age and i.female so that margins treats them this way.
* Trust (columns 1-2) excludes session 1.
* Wald test: after each model is fit, and before margins replaces the
* estimates, test that the future-entrepreneur and entrepreneur
* coefficients (2.entv2_c, 3.entv2_c) are jointly zero.  Probit and
* ordered probit report chi2(2) directly.  After tobit Stata reports the
* same Wald statistic in F form, F = chi2/2, so chi2 = 2F and p comes from
* chi2(2).  The test is on the model coefficients, not the marginal
* effects.
* Entrepreneur - future entrepreneur: after margins posts the marginal
* effects, lincom 3.entv2_c - 2.entv2_c gives the difference between the
* two marginal effects in the table, with a delta-method SE.  This equals
* ent_b - fut_b exactly.  (Re-estimating with future_entrepreneur and
* strong_entrepreneur gives the same difference in coefficients, but its
* marginal effect at means would hold future_entrepreneur at its sample
* mean, so it would not equal the difference of the reported effects.)
* Results are collected in matrix NT (rows: estimate, SE, p for future
* entrepreneur, entrepreneur, age, female; N; Wald chi2 and its p-value;
* estimate, SE, p for entrepreneur - future entrepreneur), listed at the
* end of the block.  Each set of marginal effects is also stored as nt1-nt7.

global NTX  "i.entv2_c c.age i.female i.broad_income_2 i.broad_education"
global NTME "dydx(entv2_c age female) atmeans post"

matrix NT = J(18, 7, .)
matrix colnames NT = give ret_ave contrib cont_rate request enter tourn
matrix rownames NT = fut_b fut_se fut_p ent_b ent_se ent_p age_b age_se age_p fem_b fem_se fem_p N W_chi2 W_p dif_b dif_se dif_p

capture program drop nt_store
program define nt_store
    args col n
    local r = 1
    foreach v in 2.entv2_c 3.entv2_c age 1.female {
        local r2 = `r' + 1
        local r3 = `r' + 2
        matrix NT[`r',  `col'] = _b[`v']
        matrix NT[`r2', `col'] = _se[`v']
        matrix NT[`r3', `col'] = 2*normal(-abs(_b[`v']/_se[`v']))
        local r = `r' + 3
    }
    matrix NT[13, `col'] = `n'
    * entrepreneur - future entrepreneur, from the posted marginal effects
    lincom 3.entv2_c - 2.entv2_c
    matrix NT[16, `col'] = r(estimate)
    matrix NT[17, `col'] = r(se)
    matrix NT[18, `col'] = 2*normal(-abs(r(estimate)/r(se)))
end

capture program drop nt_wald
program define nt_wald
    args col
    test 2.entv2_c 3.entv2_c
    if missing(r(chi2)) {
        local w = r(F)*r(df)
    }
    else {
        local w = r(chi2)
    }
    matrix NT[14, `col'] = `w'
    matrix NT[15, `col'] = chi2tail(2, `w')
end

* (1) trust: amount sent (0-5)
tobit give $NTX if session != 1, ll(0) ul(5) vce(robust)
local n = e(N)
nt_wald 1
margins, $NTME predict(ystar(0,5))
estimates store nt1
nt_store 1 `n'

* (2) trust: average return ratio (0-1)
tobit ret_ave $NTX if session != 1, ll(0) ul(1) vce(robust)
local n = e(N)
nt_wald 2
margins, $NTME predict(ystar(0,1))
estimates store nt2
nt_store 2 `n'

* (3) public goods: unconditional contribution (0-20)
tobit contribution $NTX, ll(0) ul(20) vce(robust)
local n = e(N)
nt_wald 3
margins, $NTME predict(ystar(0,20))
estimates store nt3
nt_store 3 `n'

* (4) public goods: conditional contribution rate (censored at 0 only)
tobit cont_rate $NTX, ll(0) vce(robust)
local n = e(N)
nt_wald 4
margins, $NTME predict(ystar(0,.))
estimates store nt4
nt_store 4 `n'

* (5) 11-20 game: amount requested.  E[request] = sum over k of k*Pr(request = k),
* built from the request values observed in the estimation sample.
oprobit request $NTX, vce(robust)
local n = e(N)
nt_wald 5
quietly levelsof request if e(sample), local(lv)
local ex ""
foreach k of local lv {
    local ex "`ex' + `k'*predict(outcome(`k'))"
}
local ex = substr("`ex'", 4, .)
margins, $NTME expression(`ex')
estimates store nt5
nt_store 5 `n'

* (6) market entry
probit enter $NTX, vce(robust)
local n = e(N)
nt_wald 6
margins, $NTME
estimates store nt6
nt_store 6 `n'

* (7) tournament entry (same specification as Table 3 column 1)
probit tournamentchoice $NTX, vce(robust)
local n = e(N)
nt_wald 7
margins, $NTME
estimates store nt7
nt_store 7 `n'

display _newline "==== NEW TABLE: marginal effect (_b), robust SE (_se), p-value (_p), N, Wald chi2(2) and p, entrepreneur - future (dif) ===="
matrix list NT, format(%9.3f)

* ---- Section 3.4.  OLS screen, measures outside the laboratory games ----
* NOT THE PAPER'S TABLE 5.  This block produced the previous Table 5 (OLS);
* the paper's Table 5 is now the "TABLE 5 -- PAPER VERSION" block directly
* below (djc 2026).  Kept for comparison with the numbers in the current
* draft and for the raw means by group.
* uniform OLS so the columns are comparable.  This is a screen, not the
* estimator of record; where Section 3.2 reports a Tobit, the Tobit governs.
* income control changed from i.broad_income to i.broad_income_2 (djc 2026)
foreach v of varlist fast_risk risk_survey1 jobrisk fast_patience eg_risk eg_skew ///
                     open extravert agreeable conscientious emotional_stability ///
                     numcorrect_eyes grit cognitive_style ///
                     search_SE plan_SE marshalling_SE implement_SE SE_all ///
                     internal_LC powerfulothers_LC chance_LC ///
                     security fear innovate social_venture ///
                     patience playertails ///
                     skilltotalcorrect correct1 correct2 correct3 {
    display _newline "==== `v' ===="
    tabstat `v', by(entv2_c) stat(n mean sd) nototal
    xi: reg `v' i.entv2_c age female i.broad_income_2 i.broad_education, robust
}

***************************************************************************
* TABLE 5 -- PAPER VERSION (djc 2026)
* Table 3 specification for the measures outside the laboratory games.
* Output: matrix T5 (the paper's table, kept rows only) and matrix T5ALL
* (all 32 measures).  Log headings: "TABLE 5 (PAPER)" and "TABLE 5, ALL 32
* MEASURES".  With P3FINAL = 1 (v4) the five online measures use all 95
* subjects with online data (see PHASE 3 FIX at the top of the file), and
* (v5) the eyes test uses the ent_final2.dta scores (EYES-TEST FIX).
***************************************************************************
* Every measure in the OLS screen above, re-estimated with the Table 3
* column (1) specification used in the NEW TABLE block: classification,
* age, female, income (i.broad_income_2) and education ($NTX), robust
* standard errors, marginal effects at the covariate means ($NTME), the
* Wald test that both classification coefficients are zero, and the
* entrepreneur - future entrepreneur difference in marginal effects with a
* delta-method SE.
* The estimator follows the outcome, as in Table 3:
*   oprobit - single-item integer scales and small counts, like the 11-20
*             request in Table 3 column (5): survey risk and patience
*             (0-10), the three job-risk items (1-7), incentivized patience
*             (0-10), honesty (0-3), Raven's matrices (0-12).  Marginal
*             effect on E[y] = sum over k of k*Pr(y = k).
*   tobit   - averaged scales and multi-item scores, censored at the scale
*             limits, like Table 3 columns (1)-(3): Eckel-Grossman risk and
*             skew (1-6), TIPI (1-7), eyes test (0-36), grit (1-5),
*             cognitive style (0-37), self-efficacy (1-7), Levenson locus of
*             control (0-48).  The three tournament-task scores (correct1-3)
*             have no upper limit and are censored at 0 only, like the
*             conditional contribution rate in Table 3 column (4).
*             Marginal effect on E[y].
*   probit  - binary items (fear of failure, innovate, social venture),
*             like Table 3 columns (6)-(7).  Marginal effect on Pr(y = 1).
* Scale limits are the instruments' limits; all are consistent with the
* observed minima and maxima in tables/summary_stats.log (25 Sep 2026).
* For each Tobit the log reports how many observations sit at each limit,
* and how many fall outside the limits (should be zero).
* Results for all 32 measures are in matrix T5ALL, one row per measure.
* The paper's Table 5 is matrix T5: only the rows where the future-
* entrepreneur coefficient, the entrepreneur coefficient, or the Wald test
* has p < $T5CUT.  Columns of both matrices:
*   m_non m_fut m_ent    unadjusted group means, on the estimation sample
*   fut_b fut_se fut_p   future entrepreneur: marginal effect, SE, p
*   ent_b ent_se ent_p   entrepreneur: marginal effect, SE, p
*   dif_b dif_se dif_p   entrepreneur - future entrepreneur
*   W_chi2 W_p           Wald test, chi2(2) and p
*   N
*   age_* fem_*          marginal effects of the controls
*   est                  estimator: 1 tobit, 2 oprobit, 3 probit

global T5CUT 0.10

local t5vars fast_risk risk_survey1 jobrisk fast_patience eg_risk eg_skew ///
             open extravert agreeable conscientious emotional_stability ///
             numcorrect_eyes grit cognitive_style ///
             search_SE plan_SE marshalling_SE implement_SE SE_all ///
             internal_LC powerfulothers_LC chance_LC ///
             security fear innovate social_venture ///
             patience playertails ///
             skilltotalcorrect correct1 correct2 correct3

* estimator and censoring limits for each measure (. = no upper limit)
foreach v in fast_risk risk_survey1 jobrisk fast_patience security patience playertails skilltotalcorrect {
    local est_`v' "oprobit"
}
foreach v in fear innovate social_venture {
    local est_`v' "probit"
}
foreach v in eg_risk eg_skew {
    local est_`v' "tobit"
    local lo_`v' 1
    local hi_`v' 6
}
foreach v in open extravert agreeable conscientious emotional_stability ///
             search_SE plan_SE marshalling_SE implement_SE SE_all {
    local est_`v' "tobit"
    local lo_`v' 1
    local hi_`v' 7
}
foreach v in internal_LC powerfulothers_LC chance_LC {
    local est_`v' "tobit"
    local lo_`v' 0
    local hi_`v' 48
}
foreach v in correct1 correct2 correct3 {
    local est_`v' "tobit"
    local lo_`v' 0
    local hi_`v' .
}
local est_numcorrect_eyes "tobit"
local lo_numcorrect_eyes  0
local hi_numcorrect_eyes  36
local est_grit            "tobit"
local lo_grit             1
local hi_grit             5
local est_cognitive_style "tobit"
local lo_cognitive_style  0
local hi_cognitive_style  37

capture program drop t5_one
program define t5_one
    args row v est lo hi
    capture drop t5s

    * fit the model
    if "`est'" == "tobit" {
        local cens "ll(`lo')"
        if "`hi'" != "." local cens "`cens' ul(`hi')"
        capture noisily tobit `v' $NTX, `cens' vce(robust)
        local rc = _rc
        local pr "predict(ystar(`lo',`hi'))"
        local code = 1
    }
    else if "`est'" == "oprobit" {
        capture noisily oprobit `v' $NTX, vce(robust)
        local rc = _rc
        local code = 2
    }
    else {
        capture noisily probit `v' $NTX, vce(robust)
        local rc = _rc
        local pr ""
        local code = 3
    }
    if `rc' {
        display as error "`v': estimation failed (rc = `rc'); row left missing"
        exit
    }
    local n = e(N)
    gen byte t5s = e(sample)

    * censoring check for the Tobits
    if "`est'" == "tobit" {
        quietly count if t5s & `v' == `lo'
        local nlo = r(N)
        local nhi = 0
        if "`hi'" != "." {
            quietly count if t5s & `v' == `hi'
            local nhi = r(N)
        }
        quietly count if t5s & !inrange(`v', `lo', `hi')
        display "`v': `nlo' obs at lower limit `lo', `nhi' at upper limit `hi', " r(N) " outside the limits"
    }

    * unadjusted group means on the estimation sample
    forvalues g = 1/3 {
        quietly summarize `v' if t5s & entv2_c == `g'
        matrix T5ALL[`row', `g'] = r(mean)
    }

    * Wald test on the model coefficients, before margins replaces them
    quietly test 2.entv2_c 3.entv2_c
    if missing(r(chi2)) {
        local w = r(F)*r(df)
    }
    else {
        local w = r(chi2)
    }

    * ordered probit: marginal effect on E[y]
    if "`est'" == "oprobit" {
        quietly levelsof `v' if t5s, local(lv)
        local ex ""
        foreach k of local lv {
            local ex "`ex' + `k'*predict(outcome(`k'))"
        }
        local ex = substr("`ex'", 4, .)
        local pr "expression(`ex')"
    }

    margins, $NTME `pr'

    * future entrepreneur (cols 4-6), entrepreneur (7-9), age (16-18), female (19-21)
    local c = 4
    foreach t in 2.entv2_c 3.entv2_c age 1.female {
        matrix T5ALL[`row', `c']   = _b[`t']
        matrix T5ALL[`row', `c'+1] = _se[`t']
        matrix T5ALL[`row', `c'+2] = 2*normal(-abs(_b[`t']/_se[`t']))
        local c = cond(`c' == 7, 16, `c' + 3)
    }

    * entrepreneur - future entrepreneur (cols 10-12)
    quietly lincom 3.entv2_c - 2.entv2_c
    matrix T5ALL[`row', 10] = r(estimate)
    matrix T5ALL[`row', 11] = r(se)
    matrix T5ALL[`row', 12] = 2*normal(-abs(r(estimate)/r(se)))

    matrix T5ALL[`row', 13] = `w'
    matrix T5ALL[`row', 14] = chi2tail(2, `w')
    matrix T5ALL[`row', 15] = `n'
    matrix T5ALL[`row', 22] = `code'

    drop t5s
end

local t5n : word count `t5vars'
matrix T5ALL = J(`t5n', 22, .)
matrix rownames T5ALL = `t5vars'
matrix colnames T5ALL = m_non m_fut m_ent fut_b fut_se fut_p ent_b ent_se ent_p ///
                        dif_b dif_se dif_p W_chi2 W_p N age_b age_se age_p fem_b fem_se fem_p est

local i = 1
foreach v of local t5vars {
    display _newline "==== TABLE 5, Table 3 specification: `v' (`est_`v'') ===="
    t5_one `i' `v' `est_`v'' `lo_`v'' `hi_`v''
    local i = `i' + 1
}

* keep a measure if any of the three tests has p < $T5CUT
capture matrix drop T5
local t5keep ""
forvalues i = 1/`t5n' {
    local pmin = min(el(T5ALL, `i', 6), el(T5ALL, `i', 9), el(T5ALL, `i', 14))
    if `pmin' < $T5CUT {
        matrix T5 = nullmat(T5) \ T5ALL[`i', 1...]
        local t5keep "`t5keep' `: word `i' of `t5vars''"
    }
}

display _newline "==== TABLE 5, ALL 32 MEASURES (diagnostic): raw means, marginal effects (_b), robust SE (_se), p (_p), Wald chi2(2) and p, N; est 1 tobit 2 oprobit 3 probit ===="
matrix list T5ALL, format(%9.3f)

display _newline "==== TABLE 5 (PAPER): measures with p < $T5CUT on the future-entrepreneur or entrepreneur effect, or the Wald test ===="
display "kept:`t5keep'"
capture noisily matrix list T5, format(%9.3f)

* ---- Section 3.4.  survey risk coverage is differential ----
gen byte has_risk = !missing(fast_risk)
tab entv2_c has_risk, row chi2

* ---- Section 3.4.  the two risk measures barely agree ----
pwcorr fast_risk eg_risk, sig obs

* ---- Section 2.7.  online attrition by classification ----
gen byte has_online = !missing(risk1)
tab entv2_c has_online, row

***************************************************************************
* FOLLOW UP
***************************************************************************

use "$DA/ent_djc_4.dta", clear
drop if repeat_2017   // the 2017 record of the repeat participant

gen byte base_non = (entv2_c==1) if !missing(entv2_c)
gen byte moved    = (entv2_18 > 1) if !missing(entv2_18)
gen byte answered = !missing(entv2_18)

* ---- Table 4 set-up: variables shared by the diagnostic and paper versions ----
* Ordered probit of 2018 entrepreneurial status (entv2_18) on all seven
* laboratory measures from Tables 2 and 3.  The 2018 risk and patience
* measures are no longer included: they were collected at the same time as
* the outcome.
*
* Coding of 2018 status.  The marginal effects assume three ordered
* categories, lowest = non-entrepreneur, highest = entrepreneur.  Check here.
tab entv2_18
tab entv2_c entv2_18

* Bring in any baseline variables that ent_djc_4.dta does not carry.  If
* they are all present this does nothing.
local need ""
foreach v in session give ret_ave enter {
    capture confirm variable `v'
    if _rc local need "`need' `v'"
}
if "`need'" != "" {
    display "merging from ent_djc_3.dta: `need'"
    merge 1:1 subject_id using "$DA/ent_djc_3.dta", keepusing(`need') keep(master match) nogenerate
}

* Trust measures: demeaned, missing set to zero.  Session 1 ran a
* different version of the trust game, so its values are treated as
* missing whatever is stored in give and ret_ave.  The mean is taken over
* all baseline subjects with valid trust data (sessions 2-8), not just
* follow-up respondents.
foreach v in give ret_ave {
    quietly summarize `v' if session != 1
    gen double dm_`v' = `v' - r(mean) if session != 1
    replace dm_`v' = 0 if missing(dm_`v')
}
label variable dm_give    "trust: amount sent, demeaned, 0 if missing"
label variable dm_ret_ave "trust: return ratio, demeaned, 0 if missing"
* Alternative, not used: also add a dummy for missing trust data
* gen byte trust_miss = (session == 1)

* 2016 survey risk (fast_risk, the intake-survey item; NOT fast_risk18,
* the same item asked in 2018).  Demeaned over all baseline subjects with
* the item, missing set to zero, as for trust.  Built here rather than
* using the stored dm_fast_risk so it is certain which year it comes from.
quietly summarize fast_risk
gen double dm_risk16 = fast_risk - r(mean)
replace dm_risk16 = 0 if missing(dm_risk16)
label variable dm_risk16 "2016 survey risk (fast_risk), demeaned, 0 if missing"

* Diagnostics for the risk control.
* (a) coverage of the 2016 item among follow-up respondents, by baseline group
display _newline "==== fast_risk (2016) coverage among follow-up respondents ===="
count if !missing(entv2_18)
count if !missing(entv2_18) & !missing(fast_risk)
tab entv2_c if !missing(entv2_18), summarize(fast_risk) obs
* (b) 2016 vs 2018 answers to the same item, observations with both
display _newline "==== correlation of fast_risk (2016) with fast_risk18 (2018) ===="
pwcorr fast_risk fast_risk18 if !missing(fast_risk) & !missing(fast_risk18), sig obs

global T4C "i.entv2_c age female wave2 i.broad_education i.broad_income_2"
global T4M "dm_give dm_ret_ave contribution cont_rate request enter tournamentchoice"
* Same seven measures with the two binary ones as factor variables, used for
* the marginal effects.  Coefficients are identical to the plain version;
* only margins treats enter and tournamentchoice differently (discrete 0->1
* change rather than a derivative).
global T4MF "dm_give dm_ret_ave contribution cont_rate request i.enter i.tournamentchoice"
global T4ME "dm_give dm_ret_ave contribution cont_rate request enter tournamentchoice"
global T4MB "dm_give dm_ret_ave contribution cont_rate request 1.enter 1.tournamentchoice"

***************************************************************************
* TABLE 4 DIAGNOSTIC -- seven-column version.  NOT IN THE PAPER.
* Kept for diagnostic purposes.  Three of its columns (1, 6 and 2) are
* reused in the paper's Table 4 (matrix T4P, below).
***************************************************************************
* Results in matrix T4: for each measure its coefficient (or marginal
* effect), robust SE and p-value.  All columns use the full controls
* (baseline status, age, female, wave, education, income).
*   Column 1: joint model, all seven measures together (coefficients).
*   Column 2: each measure on its own, same controls (coefficients).
*   Column 3: column 1 plus the 2016 risk control (coefficients).
*   Column 4: column 1 model, average marginal effect on
*             Pr(entrepreneur in 2018), the highest category.
*   Column 5: column 1 model, average marginal effect on
*             Pr(future entrepreneur or entrepreneur in 2018)
*             = 1 - Pr(non-entrepreneur), the lowest category.
*   Column 6: column 4 rescaled to a one-SD change in the measure.
*   Column 7: column 5 rescaled to a one-SD change in the measure.
* Average marginal effects: each subject's own effect, averaged over the
* sample (no atmeans).  SEs by the delta method.  Rows 22-24 are the risk
* coefficient (column 3 only); last row is N.
matrix T4 = J(25, 7, .)
matrix colnames T4 = joint alone joint_risk ame_ent ame_fut_or_ent sd_ent sd_fut_or_ent
matrix rownames T4 = give_b give_se give_p ret_b ret_se ret_p con_b con_se con_p ///
                     crate_b crate_se crate_p req_b req_se req_p ent_b ent_se ent_p ///
                     tour_b tour_se tour_p risk_b risk_se risk_p N

* (1) joint model
oprobit entv2_18 $T4C $T4M, robust
local r = 1
foreach v of global T4M {
    matrix T4[`r',   1] = _b[`v']
    matrix T4[`r'+1, 1] = _se[`v']
    matrix T4[`r'+2, 1] = 2*normal(-abs(_b[`v']/_se[`v']))
    local r = `r' + 3
}
matrix T4[25, 1] = e(N)

* (2) each measure on its own, same controls
local r = 1
foreach v of global T4M {
    display _newline "==== `v' alone ===="
    oprobit entv2_18 $T4C `v', robust
    matrix T4[`r',   2] = _b[`v']
    matrix T4[`r'+1, 2] = _se[`v']
    matrix T4[`r'+2, 2] = 2*normal(-abs(_b[`v']/_se[`v']))
    local r = `r' + 3
}
matrix T4[25, 2] = e(N)

* (3) joint model plus 2016 survey risk
oprobit entv2_18 $T4C $T4M dm_risk16, robust
local r = 1
foreach v in $T4M dm_risk16 {
    matrix T4[`r',   3] = _b[`v']
    matrix T4[`r'+1, 3] = _se[`v']
    matrix T4[`r'+2, 3] = 2*normal(-abs(_b[`v']/_se[`v']))
    local r = `r' + 3
}
matrix T4[25, 3] = e(N)

* (4) and (5) average marginal effects from the column 1 model
oprobit entv2_18 $T4C $T4MF, robust
local n = e(N)
capture drop t4s
gen byte t4s = e(sample)
estimates store t4joint

* (4) Pr(entrepreneur in 2018): highest category, outcome(#3)
margins, dydx($T4ME) predict(outcome(#3)) post
local r = 1
foreach v of global T4MB {
    matrix T4[`r',   4] = _b[`v']
    matrix T4[`r'+1, 4] = _se[`v']
    matrix T4[`r'+2, 4] = 2*normal(-abs(_b[`v']/_se[`v']))
    local r = `r' + 3
}
matrix T4[25, 4] = `n'

* (5) Pr(future entrepreneur or entrepreneur) = 1 - Pr(lowest category)
estimates restore t4joint
margins, dydx($T4ME) expression(1 - predict(outcome(#1))) post
local r = 1
foreach v of global T4MB {
    matrix T4[`r',   5] = _b[`v']
    matrix T4[`r'+1, 5] = _se[`v']
    matrix T4[`r'+2, 5] = 2*normal(-abs(_b[`v']/_se[`v']))
    local r = `r' + 3
}
matrix T4[25, 5] = `n'

* (6) and (7) effect of a one-SD change: columns 4 and 5 times the SD of
* each continuous measure in the estimation sample (t4s).  For the trust
* measures the SD is taken among respondents with valid trust data
* (sessions 2-8), using the raw variables, so the zeros filled in for
* session 1 do not shrink it.  Market entry and tournament entry are
* binary and are left as 0->1 changes (scale factor 1).  The SE scales
* by the same factor, so p-values equal those in columns 4 and 5.
matrix T4SD = J(7, 1, 1)
matrix rownames T4SD = give ret_ave contribution cont_rate request enter tournamentchoice
matrix colnames T4SD = sd_used
local i = 1
foreach v in give ret_ave contribution cont_rate request {
    if "`v'" == "give" | "`v'" == "ret_ave" {
        quietly summarize `v' if t4s & session != 1
    }
    else {
        quietly summarize `v' if t4s
    }
    matrix T4SD[`i', 1] = r(sd)
    local i = `i' + 1
}
forvalues k = 1/7 {
    local r = 3*(`k' - 1) + 1
    local s = el(T4SD, `k', 1)
    foreach c in 6 7 {
        local src = `c' - 2
        matrix T4[`r',   `c'] = el(T4, `r',   `src') * `s'
        matrix T4[`r'+1, `c'] = el(T4, `r'+1, `src') * `s'
        matrix T4[`r'+2, `c'] = el(T4, `r'+2, `src')
    }
}
matrix T4[25, 6] = `n'
matrix T4[25, 7] = `n'

display _newline "==== TABLE 4 DIAGNOSTIC (not in paper): cols 1-3 oprobit coefficients, cols 4-5 average marginal effects, cols 6-7 per-SD effects; robust SE (_se), p-value (_p), N ===="
matrix list T4, format(%9.3f)
display _newline "==== SDs used for columns 6-7 (1 = binary, not rescaled) ===="
matrix list T4SD, format(%9.3f)

* ---- Lasso, lambda chosen by BIC.  DIAGNOSTIC, NOT IN THE PAPER (djc 2026) ----
* Purpose: check whether the Table 4 model has too many right-hand-side
* variables.  Lasso cannot be combined with an ordered probit, so this is
* a linear lasso on 2018 status (entv2_18), same 66 observations (t4s),
* same regressors as T4 column 1.  Requires Stata 17 or later for
* selection(bic).  Lasso standardizes the regressors internally, so the
* penalty treats a 0-20 contribution and a 0/1 entry choice alike.
*   (a) Controls always included (in parentheses, not penalized); lasso
*       chooses only among the seven laboratory measures.
*   (b) Everything penalized, controls included.
* For each: the BIC path (lassoknots), the selected variables with their
* standardized lasso coefficients, and the post-selection OLS
* coefficients.  Post-selection coefficients are for description only;
* their usual standard errors are not valid after selection.
* Result (4 Oct 2026): (b) keeps baseline status, education, conditional
* contribution rate and trust sent; (a) keeps only the conditional rate.
* This motivated the reduced-controls model in the paper's Table 4.

* (a) controls forced in
lasso linear entv2_18 ($T4C) $T4M if t4s, selection(bic)
estimates store las_a
lassoknots
lassocoef las_a, display(coef, standardized)
lassocoef las_a, display(coef, postselection)

* (b) everything penalized
lasso linear entv2_18 $T4C $T4M if t4s, selection(bic)
estimates store las_b
lassoknots
lassocoef las_b, display(coef, standardized)
lassocoef las_b, display(coef, postselection)

* side by side: which variables each version keeps
lassocoef las_a las_b

* ---- Lasso, VERSION 2: adds the 32 Table 5 measures.  DIAGNOSTIC, NOT IN THE PAPER (djc 2026) ----
* Same as the lasso above -- linear lasso on 2018 status (entv2_18), lambda
* by BIC, same 66 observations (t4s), versions (a) controls forced in and
* (b) everything penalized -- but the candidate set is the seven laboratory
* measures ($T4M) PLUS the 32 measures screened for Table 5 (the list in
* the "TABLE 5 -- PAPER VERSION" block).  In (a) the controls are forced
* in and the 7 + 32 measures are penalized; in (b) everything is penalized.
* Results are stored as las_a2 and las_b2 and compared with las_a and
* las_b at the end of the block.
*
* Missing data.  Many of the 32 measures are missing for part of the
* sample (online measures, the second survey sitting, the job-risk items
* not asked in wave 2).  Lasso drops any observation with a missing
* regressor, so each measure is demeaned and its missing values set to
* zero, as for trust and 2016 risk in Table 4.  The mean is taken over all
* baseline subjects with the item, not just follow-up respondents.  No
* missing-data dummies are added.  The new variables are prefixed l5_.
*
* Note: SE_all is the mean of the four self-efficacy subscales, so where
* all four are observed it is collinear with them.  Lasso will normally
* keep at most one of the set; if the post-selection OLS keeps more than
* one, Stata will drop one as collinear.
*
* Variables come from ent_djc_4.dta where present; anything it does not
* carry is merged from ent_djc_3.dta, as for the trust variables above.
* eg_risk, eg_skew and SE_all are built here the same way as in the cross
* section.

local L5BASE fast_risk risk_survey1 jobrisk fast_patience ///
             risk1 risk2 risk3 risk4 skewchoice1 skewchoice2 skewchoice3 ///
             open extravert agreeable conscientious emotional_stability ///
             numcorrect_eyes grit cognitive_style ///
             search_SE plan_SE marshalling_SE implement_SE ///
             internal_LC powerfulothers_LC chance_LC ///
             security fear innovate social_venture ///
             patience playertails ///
             skilltotalcorrect correct1 correct2 correct3
local need ""
foreach v of local L5BASE {
    capture confirm variable `v'
    if _rc local need "`need' `v'"
}
if "`need'" != "" {
    display "lasso version 2, merging from ent_djc_3.dta: `need'"
    merge 1:1 subject_id using "$DA/ent_djc_3.dta", keepusing(`need') keep(master match) nogenerate
}
* Phase 3 fix (v4): fill the online measures from ent_final2.dta here too,
* and rebuild eg_risk and eg_skew from the filled components.
if $P3FINAL {
    display _newline "==== PHASE 3 FIX (follow-up file): filling online measures from ent_final2.dta ===="
    p3_fill
    capture drop eg_risk
    capture drop eg_skew
}
capture confirm variable eg_risk
if _rc egen eg_risk = rowmean(risk1 risk2 risk3 risk4)
capture confirm variable eg_skew
if _rc egen eg_skew = rowmean(skewchoice1 skewchoice2 skewchoice3)
capture confirm variable SE_all
if _rc egen SE_all  = rowmean(search_SE plan_SE marshalling_SE implement_SE)

* the 32 Table 5 measures, same order as in the Table 5 block
local L5VARS fast_risk risk_survey1 jobrisk fast_patience eg_risk eg_skew ///
             open extravert agreeable conscientious emotional_stability ///
             numcorrect_eyes grit cognitive_style ///
             search_SE plan_SE marshalling_SE implement_SE SE_all ///
             internal_LC powerfulothers_LC chance_LC ///
             security fear innovate social_venture ///
             patience playertails ///
             skilltotalcorrect correct1 correct2 correct3

* demean over all baseline subjects with the item, missing set to zero
global L5M ""
foreach v of local L5VARS {
    quietly summarize `v'
    gen double l5_`v' = `v' - r(mean)
    replace l5_`v' = 0 if missing(l5_`v')
    label variable l5_`v' "`v', demeaned, 0 if missing (lasso version 2)"
    global L5M "$L5M l5_`v'"
}

* how much of the follow-up sample is zero-filled on each measure
display _newline "==== lasso version 2: observations in t4s missing each measure (zero-filled) ===="
foreach v of local L5VARS {
    quietly count if t4s & missing(`v')
    display %-22s "`v'" %5.0f r(N)
}

* (a2) controls forced in; laboratory and Table 5 measures penalized
lasso linear entv2_18 ($T4C) $T4M $L5M if t4s, selection(bic)
estimates store las_a2
lassoknots
lassocoef las_a2, display(coef, standardized)
lassocoef las_a2, display(coef, postselection)

* (b2) everything penalized
lasso linear entv2_18 $T4C $T4M $L5M if t4s, selection(bic)
estimates store las_b2
lassoknots
lassocoef las_b2, display(coef, standardized)
lassocoef las_b2, display(coef, postselection)

* side by side: which variables each version keeps, versions 1 and 2
display _newline "==== LASSO VERSIONS 1 AND 2: variables selected (las_a, las_b = version 1; las_a2, las_b2 = version 2) ===="
lassocoef las_a las_b las_a2 las_b2

***************************************************************************
* TABLE 4 -- PAPER VERSION (djc 2026)
***************************************************************************
* Four columns, all with the seven laboratory measures:
*   Column 1: joint ordered probit, full controls (= T4 column 1),
*             coefficients.
*   Column 2: effect of a one-SD change on Pr(entrepreneur in 2018) from
*             the column 1 model (= T4 column 6), average marginal effects.
*   Column 3: each measure on its own, full controls (= T4 column 2),
*             coefficients.
*   Column 4: joint ordered probit with reduced controls -- baseline
*             status and education only (age, female, wave and income
*             dropped), coefficients.
* Full controls are baseline status, age, female, wave, education and
* income.  Rows: coefficient (or effect), robust SE and p-value for each
* measure, then N.

* (4) joint model, reduced controls
global T4CR "i.entv2_c i.broad_education"
oprobit entv2_18 $T4CR $T4M, robust

matrix T4P = J(22, 4, .)
matrix colnames T4P = joint sd_ent alone joint_reduced
matrix rownames T4P = give_b give_se give_p ret_b ret_se ret_p con_b con_se con_p ///
                      crate_b crate_se crate_p req_b req_se req_p ent_b ent_se ent_p ///
                      tour_b tour_se tour_p N

local r = 1
foreach v of global T4M {
    matrix T4P[`r',   4] = _b[`v']
    matrix T4P[`r'+1, 4] = _se[`v']
    matrix T4P[`r'+2, 4] = 2*normal(-abs(_b[`v']/_se[`v']))
    local r = `r' + 3
}
matrix T4P[22, 4] = e(N)

* columns 1-3 copied from the diagnostic matrix T4 (columns 1, 6, 2)
forvalues r = 1/21 {
    matrix T4P[`r', 1] = el(T4, `r', 1)
    matrix T4P[`r', 2] = el(T4, `r', 6)
    matrix T4P[`r', 3] = el(T4, `r', 2)
}
matrix T4P[22, 1] = el(T4, 25, 1)
matrix T4P[22, 2] = el(T4, 25, 6)
matrix T4P[22, 3] = el(T4, 25, 2)

display _newline "==== TABLE 4 (PAPER): col 1 joint coefs, col 2 per-SD effect on Pr(ent), col 3 one-at-a-time coefs, col 4 joint coefs with reduced controls; robust SE (_se), p (_p), N ===="
matrix list T4P, format(%9.3f)

* Previous Table 4 (Krista's file, income control updated), kept for reference.
* It includes the 2018 risk and patience measures and omits trust and market entry.
* oprobit entv2_18 i.entv2_c age female wave2 i.broad_education i.broad_income_2 ///
*         cont_rate contribution request tournamentchoice fast_risk18 fast_patience18, robust
* foreach v of varlist cont_rate contribution request tournamentchoice fast_risk18 fast_patience18 {
*     oprobit entv2_18 i.entv2_c age female wave2 i.broad_education i.broad_income_2 `v', robust
* }

* ---- Section 3.3.  baseline survey risk does not predict entry ----
probit moved fast_risk age female if base_non==1, robust
margins, dydx(fast_risk)

* ---- Section 3.3.  response to the follow-up is related to tournament entry ----
* Controls match Table 3 column (1) (djc 2026).  n = 122.
probit answered tournamentchoice i.entv2_c age female i.broad_income_2 i.broad_education, robust
* Krista's version, age and gender only (p = 0.037 on tournamentchoice):
* probit answered tournamentchoice age female, robust

* ---- Section 4.  how many baseline non-entrepreneurs moved ----
count if base_non==1 & !missing(entv2_18)
tab moved if base_non==1

* ---- Section 4.  stability of the repeated single-item measures ----
pwcorr fast_risk fast_risk18, sig obs
pwcorr fast_patience fast_patience18, sig obs
ttest fast_risk == fast_risk18

log close