Discuss the differences and similarities between MPI and PVM Discuss the benefits and?
MPI
(Message
P
assing
In
terface)
is
sp
eciØcation
for
message-passing
libraries
that
can
b
e
used
for
writing
p
ortable
parallel
programs.
What
do
es
MPI
do?
When
w
e
sp
eak
ab
out
parallel
programming
using
MPI,
w
e
imply
that:
≤
A
Øxed
set
of
pro
cesses
is
created
at
program
initialization,
one
pro
cess
is
created
p
er
pro
cessor
≤
Eac
h
pro
cess
kno
ws
its
p
ersonal
n
um
b
er
≤
Eac
h
pro
cess
kno
ws
n
um
b
er
of
all
pro
cesses
≤
Eac
h
pro
cess
can
comm
unicate
with
other
pro
cesses
≤
Pro
cess
can't
create
new
pro
cesses
(in
MPI{1),
the
group
of
pro
cesses
is
static
What
is
PVM?
PVM
(P
arallel
Virtual
Mac
hine)
is
a
soft
w
are
pac
k
age
that
allo
ws
a
heterogeneous
collection
of
w
orkstations
(host
p
o
ol)
to
function
as
a
single
high
p
erformance
parallel
virtual
mac
hine.
PVM,
through
its
virtual
mac
hine,
pro
vides
a
simple
y
et
useful
distributed
op
erating
system.
It
has
daemon
running
on
all
computers
making
up
the
virtual
mac
hine.
PVM
daemon
(p
vmd)
is
UNIX
pro
cess,
whic
h
o
v
ersees
the
op
eration
of
user
pro
cesses
within
a
PVM
application
and
co
ordinates
in
ter-mac
hine
PVM
comm
unications.
Suc
h
p
vmd
serv
es
as
a
message
router
and
con
troller.
One
p
vmd
runs
on
eac
h
host
of
a
virtual
mac
hine,
the
Ørst
p
vmd,
whic
h
is
started
b
y
hand,
is
designated
the
master,
while
the
others,
started
b
y
the
master,
are
called
sla
v
es.
It
means,
that
in
con
trast
to
MPI,
where
master
and
sla
v
es
start
sim
ultaneously
,
in
PVM
master
m
ust
b
e
started
on
our
lo
cal
mac
hine
and
then
it
automatically
starts
daemons
on
all
other
mac
hines.
In
PVM
only
the
master
can
start
new
sla
v
es
and
add
them
to
conØguration
7
or
delete
sla
v
e
hosts
from
the
mac
hine.
Eac
h
daemon
main
tains
a
table
of
conØguration
and
handles
information
relativ
e
to
our
parallel
virtual
mac
hine.
Pro
cesses
comm
unicate
with
eac
h
other
through
the
daemons:
they
talk
to
their
lo
cal
daemon
via
the
library
in
terface
routines,
and
lo
cal
daemon
then
sends/receiv
es
messages
to/from
remote
host
daemons.
General
idea
of
using
MPI
and
PVM
is
the
follo
wing:
The
user
writes
his
application
as
a
collection
of
co
op
erating
pro
cesses
(tasks),
that
can
b
e
p
er-
formed
indep
enden
tly
in
diÆeren
t
pro
cessors.
Pro
cesses
access
PVM/MPI
resources
through
a
library
of
standard
in
terface
routines.
These
routines
allo
w
the
initiation
and
termination
of
pro-
cesses
across
the
net
w
ork
as
w
ell
as
comm
unication
b
et
w
een
pro
cesses.
3.3
What
is
not
diÆeren
t?
Despite
their
diÆerences,
PVM
and
MPI
certainly
ha
v
e
features
in
common.
In
this
section
w
e
review
some
of
the
similarities.
3.3.1
P
ortabilit
y
Both
PVM
and
MPI
are
p
ortable;
the
sp
eciØcation
of
eac
h
is
mac
hine
indep
enden
t,
and
im-
plemen
tations
are
a
v
ailable
for
a
wide
v
ariet
y
of
mac
hines.
P
ortabilit
y
means,
that
source
co
de
written
for
one
arc
hitecture
can
b
e
copied
to
a
second
arc
hitecture,
compiled
and
executed
without
mo
diØcation.
3.3.2
MPMD
Both
MPI
and
PVM
p
ermit
diÆeren
t
pro
cesses
of
a
parallel
program
to
execute
diÆeren
t
exe-
cutable
binary
Øles
(This
w
ould
b
e
required
in
a
heterogeneous
implemen
tation,
in
an
y
case).
That
is,
b
oth
PVM
and
MPI
supp
ort
MPMD
programs
as
w
ell
as
SPMD
programs,
although
again
some
implemen
tation
ma
y
not
do
so
(MPICH,
LAM
{
supp
ort).
3.3.3
In
terop
erabilit
y
The
next
issue
is
in
terop
erabilit
y
{
the
abilit
y
of
diÆeren
t
implemen
tations
of
the
same
sp
eciØ-
cation
to
exc
hange
messages.
F
or
b
oth
PVM
and
MPI,
v
ersions
of
the
same
implemen
tation
(Oak
Ridge
PVM,
MPICH,
or
LAM)
are
in
terop
erable.
3.3.4
Heterogeneit
y
The
next
imp
ortan
t
p
oin
t
is
supp
ort
for
heterogeneit
y
.
When
w
e
wish
to
exploit
a
collection
of
net
w
ork
ed
computers,
w
e
ma
y
ha
v
e
to
con
tend
with
sev
eral
diÆeren
t
t
yp
es
of
heterogeneit
y
[GBD
+
94]:
≤
arc
hitecture
The
set
of
computers
a
v
ailable
can
include
a
wide
range
of
arc
hitecture
t
yp
es
suc
h
as
PC
class
mac
hines,
high-p
erformance
w
orkstations,
shared-memory
m
ultipro
cessors,
v
ector
sup
ercom-
puters,
and
ev
en
large
MPPs.
Eac
h
arc
hitecture
t
yp
e
has
its
o
wn
optimal
programming
metho
d.
Ev
en
when
the
arc
hitectures
are
only
serial
w
orkstations,
there
is
still
the
prob-
lem
of
incompatible
binary
formats
and
the
need
to
compile
a
parallel
task
on
eac
h
diÆeren
t
mac
hine.
8
≤
data
format
Data
formats
on
diÆeren
t
computers
are
often
incompatible.
This
incompatibilit
y
is
an
imp
or-
tan
t
p
oin
t
in
distributed
computing
b
ecause
data
sen
t
from
one
computer
ma
y
b
e
unreadable
on
the
receiving
computer.
Message
passing
pac
k
ages
dev
elop
ed
for
heterogeneous
en
viron-
men
ts
m
ust
mak
e
sure
all
the
computers
understand
the
exc
hanged
data;
they
m
ust
include
enough
information
in
the
message
to
enco
de
or
deco
de
it
for
an
y
other
computer.
≤
computational
sp
eed
Ev
en
if
the
set
of
computers
are
all
w
orkstations
with
the
same
data
format,
there
is
still
heterogeneit
y
due
to
diÆeren
t
computational
sp
eeds.
The
problem
of
computational
sp
eeds
can
b
e
v
ery
subtle.
The
programmer
m
ust
b
e
careful
that
one
w
orkstation
do
esn't
sit
idle
w
aiting
for
the
next
data
from
the
other
w
orkstation
b
efore
con
tin
uing.
≤
mac
hine
load
Our
cluster
can
b
e
comp
osed
of
a
set
of
iden
tical
w
orkstations.
But
since
net
w
ork
ed
com-
puters
can
ha
v
e
sev
eral
other
users
on
them
running
a
v
ariet
y
of
jobs,
the
mac
hine
load
can
v
ary
dramatically
.
The
result
is
that
the
eÆectiv
e
computational
p
o
w
er
across
iden
tical
w
orkstations
can
v
ary
b
y
an
order
of
magnitude.
≤
net
w
ork
load
Lik
e
mac
hine
load,
the
time
it
tak
es
to
send
a
message
o
v
er
the
net
w
ork
can
v
ary
dep
ending
on
the
net
w
ork
load
imp
osed
b
y
all
the
other
net
w
ork
users,
who
ma
y
not
ev
en
b
e
using
an
y
of
the
computers
in
v
olv
ed
in
our
computation.
This
sending
time
b
ecomes
imp
ortan
t
when
a
task
is
sitting
idle
w
aiting
for
a
message,
and
it
is
ev
en
more
imp
ortan
t
when
the
parallel
algorithm
is
sensitiv
e
to
message
arriv
al
time.
Th
us,
in
distributed
computing,
heterogeneit
y
can
app
ear
dynamically
in
ev
en
simple
setups.
Both
PVM
and
MPI
pro
vide
supp
ort
for
heterogeneit
y
.
As
for
MPI,
diÆeren
t
datat
yp
es
can
b
e
encapsulated
in
a
single
deriv
ed
t
yp
e,
thereb
y
allo
wing
comm
unication
of
heterogeneous
messages.
In
addition,
data
can
b
e
sen
t
from
one
arc
hitecture
to
another
with
data
con
v
ersion
in
heterogeneous
net
w
orks
(big-endian,
little-endian).
Although
MPI
sp
eciØcation
is
designed
to
encourage
heterogeneous
implemen
tation,
some
implemen
tations
of
MPI
ma
y
not
b
e
used
in
a
heterogeneous
en
vironmen
t.
Both
the
MPICH
and
LAM
are
implemen
tations
of
MPI,
whic
h
supp
ort
heterogeneous
en
vironmen
ts.
The
PVM
system
supp
orts
heterogeneit
y
in
terms
of
mac
hines,
net
w
orks,
and
applications.
With
regard
to
message
passing,
PVM
p
ermits
messages
con
taining
more
than
one
datat
yp
e
to
b
e
exc
hanged
b
et
w
een
mac
hines
ha
ving
diÆeren
t
data
represen
tations.
In
summary
,
b
oth
PVM
and
MPI
are
systems
designed
to
pro
vide
users
with
libraries
for
writing
p
ortable,
heterogeneous,
MPMD
programs.
3.4
DiÆerences
PVM
is
built
around
the
concept
of
a
virtual
mac
hine
whic
h
is
a
dynamic
collection
of
(p
oten-
tially
heterogeneous)
computational
resources
managed
as
a
single
parallel
computer.
The
virtual
mac
hine
concept
is
fundamen
tal
to
the
PVM
p
ersp
ectiv
e
and
pro
vides
the
basis
for
heterogeneit
y
,
p
ortabilit
y
,
and
encapsulation
of
function
that
constitute
PVM.
In
con
trast,
MPI
has
fo
cused
on
message-passing
and
explicitly
states
that
resource
managemen
t
and
the
concept
of
a
virtual
mac
hine
are
outside
the
scop
e
of
the
MPI
(1
and
2)
standard
[GKP96
].
9
3.4.1
Pro
cess
Con
trol
Pro
cess
con
trol
refers
to
the
abilit
y
to
start
and
stop
tasks,
to
Ønd
out
whic
h
tasks
are
running,
and
p
ossibly
where
they
are
running.
PVM
con
tains
all
of
these
capabilities
{
it
can
spa
wn/kill
tasks
dynamically
.
In
con
trast
MPI
{1
has
no
deØned
metho
d
to
start
new
task.
MPI{2
con
tains
functions
to
start
a
group
of
tasks
and
to
send
a
kill
signal
to
a
group
of
tasks
[NS02].
3.4.2
Resource
Con
trol
In
terms
of
resource
managemen
t,
PVM
is
inheren
tly
dynamic
in
nature.
Computing
resources
or
"hosts"
can
b
e
added
and
deleted
at
will,
either
from
a
system
"console"
or
ev
en
from
within
the
user's
application.
Allo
wing
applications
to
in
teract
with
and
manipulate
their
computing
en
vironmen
t
pro
vides
a
p
o
w
erful
paradigm
for
≤
load
balancing
|
when
w
e
w
an
t
to
reduce
idle
time
for
eac
h
mac
hine
in
v
olv
ed
in
computation
≤
task
migration
|
user
can
request
that
certain
tasks
execute
on
mac
hines
with
particular
data
formats,
arc
hitectures,
or
ev
en
on
an
explicitly
named
mac
hine
≤
fault
tolerance
Another
asp
ect
of
virtual
mac
hine
dynamics
relates
to
e±ciency
.
User
applications
can
exhibit
p
oten
tially
c
hanging
computational
needs
o
v
er
the
course
of
their
execution.
F
or
example,
con-
sider
a
t
ypical
application
whic
h
b
egins
and
ends
with
primarily
serial
computations,
but
con
tains
sev
eral
phases
of
hea
vy
parallel
computation.
PVM
pro
vides
∞exible
con
trol
o
v
er
the
amoun
t
of
computational
p
o
w
er
b
eing
utilized.
Additional
hosts
can
b
e
added
just
for
those
p
ortions
when
w
e
need
them.
MPI
lac
ks
suc
h
dynamics
and
is,
in
fact,
sp
eciØcally
designed
to
b
e
static
in
nature
to
impro
v
e
p
erformance.
Because
all
MPI
tasks
are
alw
a
ys
presen
t,
there
is
no
need
for
an
y
time-consuming
lo
okups
for
group
mem
b
ership.
Eac
h
task
already
kno
ws
ab
out
ev
ery
other
task,
and
all
com-
m
unications
can
b
e
made
without
the
explicit
need
for
a
sp
ecial
daemon.
Because
all
p
oten
tial
comm
unication
paths
are
kno
wn
at
startup,
messages
can
also,
where
p
ossible,
b
e
directly
routed
o
v
er
custom
task-to-task
c
hannels.
3.4.3
Virtual
T
op
ology
On
the
other
hand,
although
MPI
do
es
not
ha
v
e
a
concept
of
a
virtual
mac
hine,
MPI
do
es
pro
vide
a
higher
lev
el
of
abstraction
on
top
of
the
computing
resources
in
terms
of
the
message-
passing
top
ology
.
In
MPI
a
group
of
tasks
can
b
e
arranged
in
a
sp
eciØc
logical
in
terconnection
top
ology
[NS02,
F
or94]
.
A
virtual
top
ology
is
a
mec
hanism
for
naming
the
pro
cesses
in
a
group
in
a
w
a
y
that
Øts
the
comm
unication
pattern
b
etter.
The
main
aim
of
this
is
to
mak
e
subsequen
t
co
de
simpler.
It
ma
y
also
pro
vide
hin
ts
to
the
run-time
system
whic
h
allo
w
it
to
optimize
the
comm
unication
or
ev
en
hin
t
to
the
loader
ho
w
to
conØgure
the
pro
cesses.
F
or
example,
if
our
pro
cesses
will
comm
unicate
mainly
with
nearest
neigh
b
ours
after
the
fashion
of
a
t
w
o-dimensional
grid
(see
Figure
3),
w
e
could
create
a
virtual
top
ology
to
re∞ect
this
fact.
What
w
e
gain
from
this
creation
is
access
to
con
v
enien
t
routines
whic
h,
for
example,
compute
the
rank
of
an
y
pro
cess
giv
en
its
co
ordinates
in
the
grid,
taking
prop
er
accoun
t
of
b
oundary
conditions.
In
particular,
there
are
routines
to
compute
the
ranks
of
our
nearest
neigh
b
ours.
The
rank
can
then
b
e
used
as
an
argumen
t
to
message{passing
op
erations.
10
